Implementing K-means clustering for Hadoop in R and Java

October 14, 2011

(This article was first published on Revolutions, and kindly contributed to R-bloggers)

At the Bay Area R User Group meeting this week, Antonio Piccolboni gave an overview of the design goals and implementation of the RHadoop Project packages that connect Hadoop and R: rhdfs, rhbase and rmr:

RHadoop packages slide

(The image above was captured from Antionio's slides.) The most revealing part of the talk for me was the comparison of implementing the K-means clustering algorithm the "standard" way (using Python, Pig and Java, as shown on slides 8-10) compared to using just R (with the rmr package, shown on slides 14-15): it takes much less code, and can be implemented in a single language.  Antonio expands on this example at the RHadoop wiki, which makes for a great place to start if you're looking to implement big-data statistical models with the rmr package.

RHadoop wiki: Comparison of high level languages for mapreduce: k means

To leave a comment for the author, please follow the link and comment on their blog: Revolutions. offers daily e-mail updates about R news and tutorials on topics such as: Data science, Big Data, R jobs, visualization (ggplot2, Boxplots, maps, animation), programming (RStudio, Sweave, LaTeX, SQL, Eclipse, git, hadoop, Web Scraping) statistics (regression, PCA, time series, trading) and more...

If you got this far, why not subscribe for updates from the site? Choose your flavor: e-mail, twitter, RSS, or facebook...

Tags: , ,

Comments are closed.

Recent popular posts


Mango solutions

RStudio homepage

Zero Inflated Models and Generalized Linear Mixed Models with R

Dommino data lab

Quantide: statistical consulting and training



CRC R books series

Six Sigma Online Training

Contact us if you wish to help support R-bloggers, and place your banner here.

Never miss an update!
Subscribe to R-bloggers to receive
e-mails with the latest R posts.
(You will not see this message again.)

Click here to close (This popup will not appear again)