Announcing Big Data for Revolution R

[This article was first published on Revolutions, and kindly contributed to R-bloggers]. (You can report issue about the content on this page here)
Want to share your content on R-bloggers? click here if you have a blog, or here if you don't.

I’ve hinted this was coming a few times before, but with today’s press release the announcement is official: the next release of Revolution R Enterprise will include “Big Data” capabilities thanks to the new RevoScaleR package. We’re pretty excited at how it’s turned out: it’s kinda amazing to be able to use R’s formula syntax like this:

arrDelayLm2 <- rxLinMod(ArrDelay ~ DayOfWeek:F(CRSDepTime))

and be able to do a regression on 120+million rows (more than 13Gb) of data in just a few seconds using an ordinary laptop. With the more powerful multicore machines now on the market, the parallel processing algorithms in RevoScaleR really scream. You can see some of the details of how the RevoScaleR package works in the white paper “Big Data Analysis with Revolution R Enterprise“, and I’ll be giving a presentation about the new Big Data capabilities of Revolution R Enterprise in a live webcast on August 25.

One aspect of the announcement that really seems to have generated attention is how you can use Revolution R Enterprise with Hadoop to process super-massive data sets. (As I write this, before the press release is even on the wires, this article by Dave Rosenberg at Cnet has already been retweeted over 100 times.) Hadoop and RevoScaleR complement each other well: like a freight train, Hadoop can do the heavy lifting of preprocessing a distributed data set to get it ready for statistical analysis, and then, like a race car, RevoScaleR fits the statistical model. We’ll be coming out very soon with a white paper authored by Saptarshi Guha (author of the the Rhipe integration between Hadoop and R) demonstrating how he used Hadoop to extract out individual conversations from packet-level VOIP data, and then used RevoScaleR to perform a regression analysis on those calls. We’ll have more information about that analysis here in the blog in the next couple of weeks.

Revolution Analytics: Revolutionary New Levels of Performance and Scalability for Big Data Analysis

To leave a comment for the author, please follow the link and comment on their blog: Revolutions. offers daily e-mail updates about R news and tutorials about learning R and many other topics. Click here if you're looking to post or find an R/data-science job.
Want to share your content on R-bloggers? click here if you have a blog, or here if you don't.

Never miss an update!
Subscribe to R-bloggers to receive
e-mails with the latest R posts.
(You will not see this message again.)

Click here to close (This popup will not appear again)