809 search results for "parallel"

By-Group Aggregation in Parallel

October 4, 2014
By
By-Group Aggregation in Parallel

Similar to the row search, by-group aggregation is another perfect use case to demonstrate the power of split-and-conquer with parallelism. In the example below, it is shown that the homebrew by-group aggregation with foreach pakage, albeit inefficiently coded, is still a lot faster than the summarize() function in Hmisc package.

Read more »

Row Search in Parallel

September 28, 2014
By
Row Search in Parallel

I’ve been always wondering whether the efficiency of row search can be improved if the whole data.frame is splitted into chunks and then the row search is conducted within each chunk in parallel. In the R code below, a comparison is done between the standard row search and the parallel row search with the FOREACH

Read more »

Post 10: Multicore parallelism in MCMC

September 24, 2014
By
Post 10: Multicore parallelism in MCMC

MCMC is by its very nature a serial algorithm -- each iteration depends on the results of the last iteration. It is, therefore, rather difficult to parallelize MCMC code so that a single chain will run more quickly by splitting … Continue reading →

Read more »

Update to resolv (0.1.2) + valgrind and R + Parallel DNS Requests with Revolution R’s ‘foreach’ and `doParallel`

August 15, 2014
By

Thanks to a blog comment by @arj, I finally ran at least one of the new Rcpp-based through valgrind (resolv) and, sure enough there were a few memory leaks which are now fixed. However, I first ran valgind with a simple test R script that just did library(stats) to get a baseline (and dust off some...

Read more »

Introducing RcppParallel: Getting R and C++ to work (some more) in parallel

July 16, 2014
By

A common theme over the last few decades was that we could afford to simply sit back and let computer (hardware) engineers take care of increases in computing speed thanks to Moore's law. That same line of thought now frequently points out that we ar...

Read more »

Implementing mclapply() on Windows: a primer on embarrassingly parallel computation on multicore systems with R

July 14, 2014
By

An easy way to run R code in parallel on a multicore system is with the mclapply() function. Unfortunately, mclapply() does not work on Windows machines because the mclapply() implementation relies on forking and Windows does not support forking. For me, this is somewhat of a headache because I am used to using mclapply(), and

Read more »

Parallel computing in R

July 1, 2014
By
Parallel computing in R

Roughly a year ago I published an article about parallel computing in R here, in which I compared computation performance among 4 packages that provide R with parallel features once R is essentially a single-thread task package. Parallel computing is incredibly useful, but not every thing worths distribute across as many cores as possible. Actually,

Read more »

sugar in parallel

June 18, 2014
By
sugar in parallel

I've been playing with parallelising Rcpp11 implementation of sugar. For example, we have a NumericVector variable x and we want to compute e.g. sqrt(exp(x)) + 2.0. With sugar, we can do: NumericVector y = sqrt(exp(x)) + 2.0 ; and this does not...

Read more »

Rth: a Flexible Parallel Computation Package for R

June 17, 2014
By
Rth:  a Flexible Parallel Computation Package for R

I’ve been mentioning here that I’ll be discussing a new package, Rth, developed by me and Drew Schmidt, the latter of pbdR fame.  It’s now ready for use!  In this post, I’ll explain what goals Rth has, and how to use it. Platform Flexibility The key feature of Rth is in the word flexible in

Read more »

Rth: a Flexible Parallel Computation Package for R

June 17, 2014
By
Rth:  a Flexible Parallel Computation Package for R

I’ve been mentioning here that I’ll be discussing a new package, Rth, developed by me and Drew Schmidt, the latter of pbdR fame.  It’s now ready for use!  In this post, I’ll explain what goals Rth has, and how to use it. Platform Flexibility The key feature of Rth is in the word flexible in

Read more »