curl 0.4 bugfix release

January 10, 2015
By

(This article was first published on OpenCPU, and kindly contributed to R-bloggers)

opencpu logo

This week curl version 0.4 appeared on CRAN. This release fixes a memory bug that was introduced in the previous version, and which could under some circumstances crash your R session. The new version is well tested and super stable. If you are using this package, updating is highly recommended.

What is curl again?

From the manual

The curl() function provides a drop-in replacement for base url() with better performance and support for http 2.0, ssl (https://, ftps://), gzip, deflate and other libcurl goodies. This interface is implemented using the RConnection API in order to support incremental processing of both binary and text streams.

Some examples from the help page illustrating https, gzip, redirects and other stuff that base url doesn’t do well:

library(curl)

# Read from a connection
con <- curl("https://httpbin.org/get")
readLines(con)

# HTTP error
curl("https://httpbin.org/status/418", "r")

# Follow redirects
readLines(curl("https://httpbin.org/redirect/3"))

# Error after redirect
curl("https://httpbin.org/redirect-to?url=http://httpbin.org/status/418", "r")

# Auto decompress Accept-Encoding: gzip / deflate (rfc2616 #14.3)
readLines(curl("http://httpbin.org/gzip"))
readLines(curl("http://httpbin.org/deflate"))

Streaming

The advantage of curl over RCurl and httr is that the connection interface allows for streaming. For example you can use readLines to download and process data line-by-line:

con <- curl("http://jeroenooms.github.io/data/diamonds.json", open = "r")
readLines(con, n = 3)
readLines(con, n = 3)
readLines(con, n = 3)
close(con)

We can combine this with stream_in from jsonlite to stream-parse sizable datasets:

library(jsonlite)
con <- gzcon(curl("https://jeroenooms.github.io/data/nycflights13.json.gz"))
nycflights <- stream_in(con)

To leave a comment for the author, please follow the link and comment on their blog: OpenCPU.

R-bloggers.com offers daily e-mail updates about R news and tutorials on topics such as: Data science, Big Data, R jobs, visualization (ggplot2, Boxplots, maps, animation), programming (RStudio, Sweave, LaTeX, SQL, Eclipse, git, hadoop, Web Scraping) statistics (regression, PCA, time series, trading) and more...



If you got this far, why not subscribe for updates from the site? Choose your flavor: e-mail, twitter, RSS, or facebook...

Comments are closed.

Sponsors

Never miss an update!
Subscribe to R-bloggers to receive
e-mails with the latest R posts.
(You will not see this message again.)

Click here to close (This popup will not appear again)