DEoptim in Parallel

March 4, 2012

(This article was first published on FOSS Trading, and kindly contributed to R-bloggers)

Running DEoptim in parallel has been on the development team’s wishlist for awhile.  It had not been a priority though, because none of us have personally needed it.  An opportunity arose when Kris Boudt approached me about collaborating to add this functionality as part of a consultancy project for a financial services firm.

We were able to add and test the functionality within a week.  The latest revision of DEoptim on R-Forge has the capability to evaluate the objective function on multiple cores using foreach.  Very CPU-intensive problems will see speed increases in approximately linear time (less communication overhead).

I gave a short presentation (PDF) on the parallel functionality at the Saint Louis R User Group meetup in February.  A longer-running version of the code used in the presentation is on R-Forge, in the file DEoptim/sandbox/largeN_doSNOW.R (revision 86).

There are a few things to keep in mind when using the parallel functionality.  I quote from the meetup presentation:

  • Data communication between nodes can overwhelm gains from processing on multiple CPUs
    • Be careful with non-varying objects
    • Exclude them from formal function arguments
  • Copy them to nodes before optimization (clusterExport)
  • If mu and sigma were formal function arguments, they would be copied to each node for all 2037 function evaluations!

Please try it and give us feedback.  R-Forge has been undergoing major updates, so please anonymously checkout the source and build it yourself if you’re unable to download the pre-built source / binaries.

To leave a comment for the author, please follow the link and comment on their blog: FOSS Trading. offers daily e-mail updates about R news and tutorials on topics such as: Data science, Big Data, R jobs, visualization (ggplot2, Boxplots, maps, animation), programming (RStudio, Sweave, LaTeX, SQL, Eclipse, git, hadoop, Web Scraping) statistics (regression, PCA, time series, trading) and more...

If you got this far, why not subscribe for updates from the site? Choose your flavor: e-mail, twitter, RSS, or facebook...

Comments are closed.


Mango solutions

RStudio homepage

Zero Inflated Models and Generalized Linear Mixed Models with R

Quantide: statistical consulting and training




CRC R books series

Six Sigma Online Training

Contact us if you wish to help support R-bloggers, and place your banner here.

Never miss an update!
Subscribe to R-bloggers to receive
e-mails with the latest R posts.
(You will not see this message again.)

Click here to close (This popup will not appear again)