Posts Tagged ‘ MapReduce ’

Computational Journalism Server Version 1.6.5 Released

May 31, 2012
By

I’ve just released version 1.6.5 of the Computational Journalism Server. This is going to be the last release for a while. Release notes: I removed CoffeeScript and Node.js. I wasn’t using them. I dropped back to Erlang R14B-1.1. Everything tes...

Read more »

New Data Science Packages Coming To Computational Journalism Server

May 30, 2012
By

I’ve just received an announcement from Michael Lang that packages BatchJobs and BatchExperiments have been added to the Comprehensive R Archive Network (CRAN). From the announcement: The package BatchJobs implements the basic objects and procedu...

Read more »

Heartbeat of a Cycling City: Bixi data at Hack/Reduce

May 8, 2012
By
Heartbeat of a Cycling City: Bixi data at Hack/Reduce

The recent Hack/Reduce hackathon in Montreal was a tonne of fun. Our team tackled a data set of consisting of Bixi (Montreal’s bicycle share system) station states at one minute temporal resolution. We used Hadoop and mapreduce to pull out some features of user behaviours. One of the things we extracted was the flux at

Read more »

RHIPE: An Interface Between Hadoop and R for Large and Complex Data Analysis

February 16, 2011
By
RHIPE: An Interface Between Hadoop and R for Large and Complex Data Analysis

RHIPE: An Interface Between Hadoop and R Presented by Saptarshi Guha About the Video: I filmed the event using LectureMaker’s live event recording technique. One special feature I add to my R video recordings is the addition of my own R source code … Continue reading →

Read more »

Abusing Amazon’s Elastic MapReduce Hadoop service… easily, from R

January 10, 2011
By
Abusing Amazon’s Elastic MapReduce Hadoop service… easily, from R

JD Long's experimental segue package makes it easy to use Amazon's Elastic MapReduce service to fire up a Hadoop cluster and use it for non-Big Data, computationally-intensive tasks. The package provides a cluster-aware version of lapply() which "just works".

Read more »

Lessons Learned from EC2

March 24, 2010
By
Lessons Learned from EC2

A week or so ago I had my first experience using someone else’s cluster on Amazon EC2. EC2 is the Amazon Elastic Compute Cloud. Users set up a virtual computing platform that runs on Amazon’s servers “in the cloud.” Amazon EC2 is not just another cluster. EC2 allows the user to create a disk image containing an operating system...

Read more »