Statistics PhD student Nathan VanHoudnos has an 8-core laptop, and by his own admission, takes "an almost unhealthy pleasure in pushing computer to its limits". It seems like he's found an outlet for this passion with the new doSMP library included with Revolution R, that allows him to use all his processors for some gnarly simulations in R:...
Posts Tagged ‘ high-performance computing ’
Making Data Work online conference
O'Reilly is hosting a conference on June 9 on the topic of the analysis of large data sets. The title of the conference is Making Data Work: Ever since Hal Varian proclaimed that data analysis is the sexy career for the coming decade, people have been talking about data. And big data. And even bigger data. This online conference,...
Map-reduce in R with Amazon EMR
There's new video at the Video Rchive of a JD Long presentation: Amazon Elastic Map-Reduce (So Easy an Economist Can Do It). I'm on a plane right now and the in-flight wi-fi isn't quite up to playing the 30-minute video, but judging from the slides it looks like a nice, practical demonstration of getting a map-reduce computation programmed in...
Prediction in the cloud: turbulent
While Microsoft rolled out its Technical Computing Initiative -- promising new tools for distributed parallel computing on large data sets in the cloud -- with much fanfare earlier this week, Google made a rather more understated response. In a post to the developer-focused Google Code Blog, they quietly announced two new, but potentially disruptive, products. Google BigQuery promises super-fast...
Parallel Computing with R for Life Sciences
I hadn't heard of the CloudAsia 2010 conference before, but from the programme the workshop Master Class on HPC Application For Life Sciences looked like it was interesting. One workshop session in particular caught my eye: Practical Parallel Computing in R by Xie Chao and Tan Tin Wee (from the National University of Singapore). The workshop notes (PDF) provide...
Slides from High-Performance Analytics webinar now available
Thanks to everyone who attended the webinar I presented this morning, High-Performance Analytics with REvolution R and Windows HPC Server. My slides are now available for download at the link below; even if you're not using Windows, I hope the slides are a useful introduction to the foreach parallel programming construct in general. If you do use R on...
Webinar: High-Performance Analytics with R and Microsoft HPC Server
On April 14 I'll be giving a new webinar in partnership with Microsoft on High-Performance Computing with R. I'll be focusing on the new parallel programming capabilities of REvolution R Enterprise 3.1 for Windows, and how to use the features of Microsoft HPC Server to enable computing on clusters. Here's the complete agenda, and you can register at the...
Speeding up simulations with Amazon EC2
Over at Cerebral Mastication, JD Long tells a characteristically entertaining and informative story about how he uses R to run stochastic simulations of insurance portfolios and reinsurance treaties. A typical job involves 10,000 simulations, and when each estimate takes over 20 seconds you're talking some serious time to get the job done. Fortunately, this is the kind of problem...
According to Microsoft, the fourth paradigm of science is data
In scientific discovery, the first three paradigms were experimental, theoretical and (more recently) computational science. A new book of essays published by Microsoft (and available for free download -- kudos, MS!) argues that a fourth paradigm of scientific discovery is at hand: the analysis of massive data sets. The book is dedicated to the late Microsoft researcher Dr Jim...
Zero Inflated Models and Generalized Linear Mixed Models with R.
Zuur, Saveliev, Ieno (2012).
