DEoptim in Parallel

[This article was first published on FOSS Trading, and kindly contributed to R-bloggers]. (You can report issue about the content on this page here)
Want to share your content on R-bloggers? click here if you have a blog, or here if you don't.

Running DEoptim in parallel has been on the development team’s wishlist for awhile.  It had not been a priority though, because none of us have personally needed it.  An opportunity arose when Kris Boudt approached me about collaborating to add this functionality as part of a consultancy project for a financial services firm.

We were able to add and test the functionality within a week.  The latest revision of DEoptim on R-Forge has the capability to evaluate the objective function on multiple cores using foreach.  Very CPU-intensive problems will see speed increases in approximately linear time (less communication overhead).

I gave a short presentation (PDF) on the parallel functionality at the Saint Louis R User Group meetup in February.  A longer-running version of the code used in the presentation is on R-Forge, in the file DEoptim/sandbox/largeN_doSNOW.R (revision 86).

There are a few things to keep in mind when using the parallel functionality.  I quote from the meetup presentation:
  • Data communication between nodes can overwhelm gains from processing on multiple CPUs
    • Be careful with non-varying objects
    • Exclude them from formal function arguments
  • Copy them to nodes before optimization (clusterExport)
  • If mu and sigma were formal function arguments, they would be copied to each node for all 2037 function evaluations!
Please try it and give us feedback.  R-Forge has been undergoing major updates, so please anonymously checkout the source and build it yourself if you’re unable to download the pre-built source / binaries.

To leave a comment for the author, please follow the link and comment on their blog: FOSS Trading.

R-bloggers.com offers daily e-mail updates about R news and tutorials about learning R and many other topics. Click here if you're looking to post or find an R/data-science job.
Want to share your content on R-bloggers? click here if you have a blog, or here if you don't.

Never miss an update!
Subscribe to R-bloggers to receive
e-mails with the latest R posts.
(You will not see this message again.)

Click here to close (This popup will not appear again)