DEoptim in Parallel
in parallel has been on the development team's wishlist for awhile. It had not been a priority though, because none of us have personally needed it. An opportunity arose when Kris Boudt
approached me about collaborating to add this functionality as part of a consultancy project for a financial services firm.
We were able to add and test the functionality within a week. The latest revision of DEoptim on R-Forge
has the capability to evaluate the objective function on multiple cores using foreach
. Very CPU-intensive problems will see speed increases in approximately linear time (less communication overhead).
I gave a short presentation
(PDF) on the parallel functionality at the Saint Louis R User Group
meetup in February. A longer-running version of the code used in the presentation is on R-Forge, in the file DEoptim/sandbox/largeN_doSNOW.R
There are a few things to keep in mind when using the parallel functionality. I quote from the meetup presentation:
- Data communication between nodes can overwhelm gains from processing on multiple CPUs
- Be careful with non-varying objects
- Exclude them from formal function arguments
- Copy them to nodes before optimization (clusterExport)
- If mu and sigma were formal function arguments, they would be copied to each node for all 2037 function evaluations!
Please try it and give us feedback. R-Forge has been undergoing major updates, so please anonymously checkout the source and build it yourself if you're unable to download the pre-built source / binaries.
To leave a comment
for the author, please follow the link and comment on his blog: FOSS Trading
offers daily e-mail updates
news and tutorials
on topics such as: visualization (ggplot2
), programming (RStudio
, Web Scraping
) statistics (regression
, time series
) and more...
If you got this far, why not subscribe for updates
from the site? Choose your flavor: e-mail
, or facebook