785 search results for "parallel"

Multicore (parallel) processing in R

August 27, 2013
By

Multicore (parallel) processing in R from Wallace Campbell on Vimeo. If you're not programming in parallel, you're only using a fraction of your computer's power! I demonstrate how to run "for" loops in parallel using the mclapply function from the multicore library. The code can be scaled to any number of available cores.

Read more »

Advanced sab-R-metrics: Parallelization with the ‘mgcv’ Package

July 18, 2013
By
Advanced sab-R-metrics: Parallelization with the ‘mgcv’ Package

Carson Sievert (creator of the really neat pitchRx package) and Steamer Projections posed a question about reasonable run times of the mgcv package on large data in R yesterday, and promised my Pitch F/X friends I would post here with a quick tip on sp...

Read more »

Parallel Random Number Generation using TRNG

July 10, 2013
By
Parallel Random Number Generation using TRNG

To my surprise and disappointment, popular scientific libraries like Boost or GSL provide no native support for parallel random number generation. Recently I came across TRNG, an excellent random number generation library for C++ built specifically with parallel architectures in mind. Over the last few days I’ve been trawling internet forums and reading discussions about The post Parallel...

Read more »

Parallel computation with helper threads in pqR

June 23, 2013
By
Parallel computation with helper threads in pqR

One innovative feature of pqR (my new, faster, version of R), is that it can perform some numeric computations in “helper” threads, in parallel with other such numeric computations, and with interpretive operations performed in the “master” thread. This can potentially speed up your computations by a factor as large as the number of processor cores

Read more »

Intro to Parallel Random Number Generation with RevoScaleR

June 6, 2013
By
Intro to Parallel Random Number Generation with RevoScaleR

by Joseph Rickert Random number generation is fundamental to doing computational statistics. As you might expect, R is very rich in random number resources. The R base code provides several high quality random number generators including: Wichmann-Hill, Marsaglia-Multicarry, Super-Duper, Mersenne-Twister, Knuth-TAOCP-2002 and L’Ecuyer-CMRG. (See Random for details.) And, there are at least three packages, rspring, rlecuyer, and rstream for...

Read more »

Grid Search for Free Parameters with Parallel Computing

June 1, 2013
By
Grid Search for Free Parameters with Parallel Computing

In my previous post (http://statcompute.wordpress.com/2013/05/25/test-drive-of-parallel-computing-with-r) on 05/25/2013, I’ve demonstrated the power of parallel computing with various R packages. However, in the real world, it is not straight-forward to utilize these powerful tools in our day-by-day computing tasks without carefully formulate the problem. In the example below, I am going to show how to use the

Read more »

Are parallel computations worth it ?

May 31, 2013
By
Are parallel computations worth it ?

Yesterday, Daniel Marcelino published an interesting post on his blog, untitled Parallel Processing: When does it worth ? I was asking myself the same question for a chapter I am currently writing. And I did like his approach, so I tried, on my computer to do the same. I did use three packages to run parallel R codes, >...

Read more »

Parallel Processing: When does it worth?

May 29, 2013
By
Parallel Processing: When does it worth?

Most computers nowadays have few cores that incredibly help us with our daily computing duties. However, when statistical softwares do use parallelization for analyzing data faster? R, my preferred analytical package, does not take too much advantage of multicore processing by default. In fact, R has been inherently a “single-processor” package until nowadays. Stata, another

Read more »

Import All Text Files in A Folder with Parallel Execution

May 26, 2013
By
Import All Text Files in A Folder with Parallel Execution

Sometimes, we might need to import all files, e.g. *.txt, with the same data layout in a folder without knowing each file name and then combine all pieces together. With the old method, we can use lapply() and do.call() functions to accomplish the task. However, when there are a large number of such files and

Read more »

Test Drive of Parallel Computing with R

May 25, 2013
By
Test Drive of Parallel Computing with R

Today, I did a test run of parallel computing with snow and multicore packages in R and compared the parallelism with the single-thread lapply() function. In the test code below, a data.frame with 20M rows is simulated in a Ubuntu VM with 8-core CPU and 10-G memory. As the baseline, lapply() function is employed to

Read more »