898 search results for "parallel"

Parallel Simulation of Heckman Selection Model

April 22, 2015
By
Parallel Simulation of Heckman Selection Model

Parallel Simulation of Heckman Selection Model One of the, if not the, fundamental problems in observational data analysis is the estimation of the value of the unobserved choice. If the (i^{text{th}}) unit chooses the value of (t) on the basis of some factors (mathbf{x_i}), which may include...

Read more »

Accelerating R with multi-node parallelism – Rmpi, BatchJobs and OpenLava

Accelerating R with multi-node parallelism –  Rmpi, BatchJobs and OpenLava

Gord Sissons, Feng Li In a previous blog we showed how we could use the R BatchJobs package with OpenLava to accelerate a single-threaded k-means calculation by breaking the workload into chunks and running  them as serial jobs. R users frequently need to find solutions to parallelize workloads, and while solutions like multicore and socket

Read more »

Coarse Grain Parallelism with foreach and rxExec

April 2, 2015
By

by Joseph Rickert I have written a several posts about the Parallel External Memory Algorithms (PEMAs) in Revolution Analytics’ RevoScaleR package, most recently about rxBTrees(), but I haven’t said much about rxExec(). rxExec() is not itself a PEMA, but it can be used to write parallel algorithms. Pre-built PEMAs such as rxBTrees(), rxLinMod(), etc are inherently parallel algorithms designed...

Read more »

Seeing the Forest and the Trees – a parallel machine learning example

April 1, 2015
By
Seeing the Forest and the Trees – a parallel machine learning example

Parallelizing Random Forests in R with BatchJobs and OpenLava By: Gord Sissons and Feng Li In his series of blogs about machine learning, Trevor Stephens focuses on a survival model from the Titanic disaster and provides a tutorial explaining how decision trees tend to over-fit models yielding anomalous predictions. How do we build a better

Read more »

Parallel R with BatchJobs

March 28, 2015
By
Parallel R with BatchJobs

Parallelizing R with BatchJobs – An example using k-means Gord Sissons, Feng Li Many simulations in R are long running. Analysis of statistical algorithms can generate workloads that run for hours if not days tying up a single computer. Given the amount of time R programmers can spend waiting for results, getting acquainted parallelism makes

Read more »

Creating progress bars with foreach parallel processing

March 10, 2015
By
Creating progress bars with foreach parallel processing

by Andrie de Vries In my previous post, I demonstrated how to get some status of running jobs on a parallel back end. However, I stopped short of actually demonstrating progress bars. In this post I demonstrate how to do this. The StackOverflow question How do you create a progress bar when using the “foreach()” function in R? ()...

Read more »

Monitoring progress of a foreach parallel job

February 24, 2015
By
Monitoring progress of a foreach parallel job

by Andrie de Vries R has strong support for parallel programming, both in base R and additional CRAN packages. For example, we have previously written about foreach and parallel programming in the articles Tutorial: Parallel programming with foreach and Intro to Parallel Random Number Generation with RevoScaleR. The foreach package provides simple looping constructs in R, similar to lapply()...

Read more »

How-to go parallel in R – basics + tips

February 16, 2015
By
How-to go parallel in R – basics + tips

Today is a good day to start parallelizing your code. I've been using the parallel package since its integration with R (v. 2.14.0) and its much easier than it at first seems. In this post I'll go through the basics for implementing parallel computations in R, cover a few common pitfalls, and give tips on how to avoid them....

Read more »

Parallel Programming with GPUs and R

January 27, 2015
By

by Norman Matloff You've heard that graphics processing units — GPUs — can bring big increases in computational speed. While GPUs cannot speed up work in every application, the fact is that in many cases it can indeed provide very rapid computation. In this tutorial, we'll see how this is done, both in passive ways (you write only R),...

Read more »

Debugging Parallel Code with dbs()

January 4, 2015
By
Debugging Parallel Code with dbs()

I mentioned yesterday that my partools package is now on CRAN.  A number of people have expressed interest in the Snowdoop section, but in this post I want to call attention to the dbs() debugging tool in the package, useful for debugging code written for the portion of R’s parallel library that came from the … Continue reading...

Read more »