961 search results for "parallel"

Creating progress bars with foreach parallel processing

March 10, 2015
By
Creating progress bars with foreach parallel processing

by Andrie de Vries In my previous post, I demonstrated how to get some status of running jobs on a parallel back end. However, I stopped short of actually demonstrating progress bars. In this post I demonstrate how to do this. The StackOverflow question How do you create a progress bar when using the “foreach()” function in R? ()...

Read more »

Monitoring progress of a foreach parallel job

February 24, 2015
By
Monitoring progress of a foreach parallel job

by Andrie de Vries R has strong support for parallel programming, both in base R and additional CRAN packages. For example, we have previously written about foreach and parallel programming in the articles Tutorial: Parallel programming with foreach and Intro to Parallel Random Number Generation with RevoScaleR. The foreach package provides simple looping constructs in R, similar to lapply()...

Read more »

How-to go parallel in R – basics + tips

February 16, 2015
By
How-to go parallel in R – basics + tips

Today is a good day to start parallelizing your code. I've been using the parallel package since its integration with R (v. 2.14.0) and its much easier than it at first seems. In this post I'll go through the basics for implementing parallel computations in R, cover a few common pitfalls, and give tips on how to avoid them....

Read more »

Parallel Programming with GPUs and R

January 27, 2015
By

by Norman Matloff You've heard that graphics processing units — GPUs — can bring big increases in computational speed. While GPUs cannot speed up work in every application, the fact is that in many cases it can indeed provide very rapid computation. In this tutorial, we'll see how this is done, both in passive ways (you write only R),...

Read more »

Debugging Parallel Code with dbs()

January 4, 2015
By
Debugging Parallel Code with dbs()

I mentioned yesterday that my partools package is now on CRAN.  A number of people have expressed interest in the Snowdoop section, but in this post I want to call attention to the dbs() debugging tool in the package, useful for debugging code written for the portion of R’s parallel library that came from the … Continue reading...

Read more »

Parallelism via “parSapply”

December 13, 2014
By

In an earlier post, I used mclapply to kick off parallel R processes and to demonstrate inter-process synchronization via the flock package. Although I have been using this approach to parallelism for a few years now, I admit, it has certain important disadvantages. It works only on a single machine, and also, it doesn’t work

Read more »

By-Group Aggregation in Parallel

October 4, 2014
By
By-Group Aggregation in Parallel

Similar to the row search, by-group aggregation is another perfect use case to demonstrate the power of split-and-conquer with parallelism. In the example below, it is shown that the homebrew by-group aggregation with foreach pakage, albeit inefficiently coded, is still a lot faster than the summarize() function in Hmisc package.

Read more »

Row Search in Parallel

September 28, 2014
By
Row Search in Parallel

I’ve been always wondering whether the efficiency of row search can be improved if the whole data.frame is splitted into chunks and then the row search is conducted within each chunk in parallel. In the R code below, a comparison is done between the standard row search and the parallel row search with the FOREACH

Read more »

Post 10: Multicore parallelism in MCMC

September 24, 2014
By
Post 10: Multicore parallelism in MCMC

MCMC is by its very nature a serial algorithm -- each iteration depends on the results of the last iteration. It is, therefore, rather difficult to parallelize MCMC code so that a single chain will run more quickly by splitting … Continue reading →

Read more »

Update to resolv (0.1.2) + valgrind and R + Parallel DNS Requests with Revolution R’s ‘foreach’ and `doParallel`

August 15, 2014
By

Thanks to a blog comment by @arj, I finally ran at least one of the new Rcpp-based through valgrind (resolv) and, sure enough there were a few memory leaks which are now fixed. However, I first ran valgind with a simple test R script that just did library(stats) to get a baseline (and dust off some...

Read more »