726 search results for "parallel"

Parallel Processing: When does it worth?

May 29, 2013
By
Parallel Processing: When does it worth?

Most computers nowadays have few cores that incredibly help us with our daily computing duties. However, when statistical softwares do use parallelization for analyzing data faster? R, my preferred analytical package, does not take too much advantage of multicore processing by default. In fact, R has been inherently a “single-processor” package until nowadays. Stata, another

Read more »

Import All Text Files in A Folder with Parallel Execution

May 26, 2013
By
Import All Text Files in A Folder with Parallel Execution

Sometimes, we might need to import all files, e.g. *.txt, with the same data layout in a folder without knowing each file name and then combine all pieces together. With the old method, we can use lapply() and do.call() functions to accomplish the task. However, when there are a large number of such files and

Read more »

Test Drive of Parallel Computing with R

May 25, 2013
By
Test Drive of Parallel Computing with R

Today, I did a test run of parallel computing with snow and multicore packages in R and compared the parallelism with the single-thread lapply() function. In the test code below, a data.frame with 20M rows is simulated in a Ubuntu VM with 8-core CPU and 10-G memory. As the baseline, lapply() function is employed to

Read more »

Automatic ARMA/GARCH selection in parallel

March 24, 2013
By

In the original ARMA/GARCH post I outlined the implementation of the garchSearch function. There have been a few requests for the code so … here it is. Quite easy to use too: After the last code line above, fit contains the best (according to the AIC statistic) model, which is the return value of garchFit.

Read more »

Load Balanced Parallelization with snowfall

March 5, 2013
By
Load Balanced Parallelization with snowfall

For some reason, I didn't notice a few months ago the best way to perform a parallelized version of Lapply with package snowfall.We implemented the parallel version of function lapply with the function sfLapply, in the development of our pipeline p...

Read more »

reading raster data using library(parallel)

March 3, 2013
By
reading raster data using library(parallel)

Recently, I have been doing some analysis for a project I am involved in. In particular, I was interested what role pacific sea surface temperatures play with regard to rainfall in East Africa. I spare you the details as I … Continue reading →

Read more »

RMark: Parallel fast merging

March 3, 2013
By

Last time we looked at a different method of merging multiple files via fast mergin g. I highly recommend checking the fast merging function before we will translate it to a parallel solution. Parallel programming in this case means we are using more t...

Read more »

Large correlation in parallel

February 24, 2013
By
Large correlation in parallel

A little improvement to the bigcor function proposed on Rmazing to compute huge correlation matrix in R, I made the function work in parallel using all the CPU cores available on the machine. The code is here.Here is a benchmark of the 2 func...

Read more »

Parallel Array Computations With SciDB and R

January 22, 2013
By

R Evangelist Bryan Lewis on a natural integration of the R analytic environment and SciDB's distributed, multidimensional array database.

Read more »

Speeding up R computations Pt III: parallelization

January 8, 2013
By

In two previous posts, I have written about how you can speed up your R computations either by using strange notation and non-standard functions or by compiling your code. Last year my department bought a 64-core computational server, which allowed me ...

Read more »