The Dallas R User Group had a meeting over the weekend. One of the discussions is the memory limitations with R. This is a common subject among the R community and R User Groups. There has been a lot of strides recently in allowing R to stretch its memory limitations. I thought I would compile and share some of the best resources I have found to remedy the big data issue.
This package allocates hard disk space to big data vectors.
This package allocates points to unused memory or points to a swap file.
Taking R to the Limit: Parallelism and Big Data
Hitting the Big DataCeiling Limit in R
While this is not a helpful article for big data it does show some of the issues R current faces. Namely the issue of that lack of a “int64” or Long Long data type memory allocation.
Revolution R Enterprise
Revolution Analytics is creating enterprise software around R to tackle issues of big data, parallelism and threaded computing in order to speed up large data processing and analytics.