DEoptim in Parallel

March 4, 2012
By

(This article was first published on FOSS Trading, and kindly contributed to R-bloggers)

Running DEoptim in parallel has been on the development team's wishlist for awhile.  It had not been a priority though, because none of us have personally needed it.  An opportunity arose when Kris Boudt approached me about collaborating to add this functionality as part of a consultancy project for a financial services firm.

We were able to add and test the functionality within a week.  The latest revision of DEoptim on R-Forge has the capability to evaluate the objective function on multiple cores using foreach.  Very CPU-intensive problems will see speed increases in approximately linear time (less communication overhead).

I gave a short presentation (PDF) on the parallel functionality at the Saint Louis R User Group meetup in February.  A longer-running version of the code used in the presentation is on R-Forge, in the file DEoptim/sandbox/largeN_doSNOW.R (revision 86).

There are a few things to keep in mind when using the parallel functionality.  I quote from the meetup presentation:
  • Data communication between nodes can overwhelm gains from processing on multiple CPUs
    • Be careful with non-varying objects
    • Exclude them from formal function arguments
  • Copy them to nodes before optimization (clusterExport)
  • If mu and sigma were formal function arguments, they would be copied to each node for all 2037 function evaluations!
Please try it and give us feedback.  R-Forge has been undergoing major updates, so please anonymously checkout the source and build it yourself if you're unable to download the pre-built source / binaries.

To leave a comment for the author, please follow the link and comment on his blog: FOSS Trading.

R-bloggers.com offers daily e-mail updates about R news and tutorials on topics such as: visualization (ggplot2, Boxplots, maps, animation), programming (RStudio, Sweave, LaTeX, SQL, Eclipse, git, hadoop, Web Scraping) statistics (regression, PCA, time series, trading) and more...



If you got this far, why not subscribe for updates from the site? Choose your flavor: e-mail, twitter, RSS, or facebook...

Comments are closed.