827 search results for "parallel"

Parallel computation with helper threads in pqR

June 23, 2013
By
Parallel computation with helper threads in pqR

One innovative feature of pqR (my new, faster, version of R), is that it can perform some numeric computations in “helper” threads, in parallel with other such numeric computations, and with interpretive operations performed in the “master” thread. This can potentially speed up your computations by a factor as large as the number of processor cores

Read more »

Intro to Parallel Random Number Generation with RevoScaleR

June 6, 2013
By
Intro to Parallel Random Number Generation with RevoScaleR

by Joseph Rickert Random number generation is fundamental to doing computational statistics. As you might expect, R is very rich in random number resources. The R base code provides several high quality random number generators including: Wichmann-Hill, Marsaglia-Multicarry, Super-Duper, Mersenne-Twister, Knuth-TAOCP-2002 and L’Ecuyer-CMRG. (See Random for details.) And, there are at least three packages, rspring, rlecuyer, and rstream for...

Read more »

Grid Search for Free Parameters with Parallel Computing

June 1, 2013
By
Grid Search for Free Parameters with Parallel Computing

In my previous post (http://statcompute.wordpress.com/2013/05/25/test-drive-of-parallel-computing-with-r) on 05/25/2013, I’ve demonstrated the power of parallel computing with various R packages. However, in the real world, it is not straight-forward to utilize these powerful tools in our day-by-day computing tasks without carefully formulate the problem. In the example below, I am going to show how to use the

Read more »

Are parallel computations worth it ?

May 31, 2013
By
Are parallel computations worth it ?

Yesterday, Daniel Marcelino published an interesting post on his blog, untitled Parallel Processing: When does it worth ? I was asking myself the same question for a chapter I am currently writing. And I did like his approach, so I tried, on my computer to do the same. I did use three packages to run parallel R codes, >...

Read more »

Parallel Processing: When does it worth?

May 29, 2013
By
Parallel Processing: When does it worth?

Most computers nowadays have few cores that incredibly help us with our daily computing duties. However, when statistical softwares do use parallelization for analyzing data faster? R, my preferred analytical package, does not take too much advantage of multicore processing by default. In fact, R has been inherently a “single-processor” package until nowadays. Stata, another

Read more »

Import All Text Files in A Folder with Parallel Execution

May 26, 2013
By
Import All Text Files in A Folder with Parallel Execution

Sometimes, we might need to import all files, e.g. *.txt, with the same data layout in a folder without knowing each file name and then combine all pieces together. With the old method, we can use lapply() and do.call() functions to accomplish the task. However, when there are a large number of such files and

Read more »

Test Drive of Parallel Computing with R

May 25, 2013
By
Test Drive of Parallel Computing with R

Today, I did a test run of parallel computing with snow and multicore packages in R and compared the parallelism with the single-thread lapply() function. In the test code below, a data.frame with 20M rows is simulated in a Ubuntu VM with 8-core CPU and 10-G memory. As the baseline, lapply() function is employed to

Read more »

Automatic ARMA/GARCH selection in parallel

March 24, 2013
By

In the original ARMA/GARCH post I outlined the implementation of the garchSearch function. There have been a few requests for the code so … here it is. Quite easy to use too: After the last code line above, fit contains the best (according to the AIC statistic) model, which is the return value of garchFit.

Read more »

Load Balanced Parallelization with snowfall

March 5, 2013
By
Load Balanced Parallelization with snowfall

For some reason, I didn't notice a few months ago the best way to perform a parallelized version of Lapply with package snowfall. We implemented the parallel version of function lapply with the function sfLapply, in the development of our pipeline p...

Read more »

reading raster data using library(parallel)

March 3, 2013
By
reading raster data using library(parallel)

Recently, I have been doing some analysis for a project I am involved in. In particular, I was interested what role pacific sea surface temperatures play with regard to rainfall in East Africa. I spare you the details as I … Continue reading →

Read more »