Speeding up simulations with Amazon EC2

February 10, 2010

(This article was first published on Revolutions, and kindly contributed to R-bloggers)

Over at Cerebral Mastication, JD Long tells a characteristically entertaining and informative story about how he uses R to run stochastic simulations of insurance portfolios and reinsurance treaties. A typical job involves 10,000 simulations, and when each estimate takes over 20 seconds you’re talking some serious time to get the job done. 

Fortunately, this is the kind of problem that lends itself to parallelization: with a multiprocessor machine running R, you can run as many simulations as you have processors at the same time, and cut the total computation time by the same factor. JD describes how he boots up an 8-core instance running R on Amazon EC2, and then uses the multicore package on Linux to reduce the computation time by a factor of 8.

Sadly for Windows users, multicore doesn’t run on Windows. But as Boris Shor points out in a comment to JD’s post, REvolution R Enterprise allows you to do parallel programming like this (using the foreach operator) on a multiprocessor Windows machine (or even a cluster of many machines).

Cerebral Mastication: Using the R multicore package in Linux with wild an passionate abandon

To leave a comment for the author, please follow the link and comment on their blog: Revolutions.

R-bloggers.com offers daily e-mail updates about R news and tutorials on topics such as: Data science, Big Data, R jobs, visualization (ggplot2, Boxplots, maps, animation), programming (RStudio, Sweave, LaTeX, SQL, Eclipse, git, hadoop, Web Scraping) statistics (regression, PCA, time series, trading) and more...

If you got this far, why not subscribe for updates from the site? Choose your flavor: e-mail, twitter, RSS, or facebook...

Tags: , ,

Comments are closed.


Never miss an update!
Subscribe to R-bloggers to receive
e-mails with the latest R posts.
(You will not see this message again.)

Click here to close (This popup will not appear again)