There are many options available with this command including the choice of repository, the dependency options,  etc….

Getting external data into R

From my personal experience there is no better way than .csv files to load data into R. The format is dead simple, flexible and you can play around with fairly large datasets without (too much) trouble. R provides a simple function to load csv files.

data <- read.csv(…)

However you might have to interface with relational databases. In that case R provides a wide range of possibilities. RODBC is a package that connects R to SQL Server, Access or Excel. Amongst other features it allows to pass SQL code within R and get the result straight back onto an R object. By the way, I’m not a Windows fan just a pragmatic user….

There are equivalent packages providing interfaces to MySQL and PostgreSQL: RMySQL and RPostgreSQL. I haven’t tested them but I never heard of bad comments from users.

Communicating with external applications

I red somewhere that R shouldn’t be seen as an island but rather a specific tool to do specific tasks. I can’t think of any better definition of my day to day job. R is unmatched when it comes to statistical analysis but you might sometimes need to call it from external applications or you might need to get data from external sources straight into R…. Whatever the task at hand there is a good chance that you will find just the tool you need. I list below some of the most useful tools I came across:

R-(D)COM is a programming interface to COM and DCOM (ex ActiveX; Microsoft distributed object interface) to access the R calculation engine. As such, it runs only under the Windows environment. Thomas Baier and Erich Neuwirth have created a website dedicated to this technology. You will find there detailed instructions about installation and a video with examples. Using the same technology Vincent Vella’s blog explains how to make R works within C#.

The RBloomberg package developed by Robert Sams is another interesting tool. It allows essentially to retrieve data from Bloomberg and put them straight into R. More details are available on the CRAN and on R-Finance mailing list.

Obviously the list doesn’t stop here but it’s beyond the scope of this article to list all of them.

Optimizing R code

In typical financial applications R code doesn’t really have to be optimized when you deal with daily or lower frequency data. However when dealing with intraday or (even worst) tick data writing proper R code makes a huge difference. There are several R tools available out there to make R code better (i.e run faster). The code bellow tells how long it takes to run a given set of R instructions

System.time([your R code])

We can go further and identify which part of your code takes longer.

Rprof("d1.prof")
[your R code]
Rprof(NULL)
d1prof = summaryRprof("d1.prof")
head(d1prof$by.self)

The output of the above code will display the time taken by each command in absolute and relative terms.

              self.time self.pct total.time total.pct
[.data.frame       0.54     26.0       0.88      42.3
==                 0.28     13.5       0.28      13.5
&                  0.16      7.7       0.16       7.7
[[.data.frame      0.08      3.8       0.22      10.6
[                  0.06      2.9       0.94      45.2
which              0.06      2.9       0.74      35.6
match              0.06      2.9       0.24      11.5
inherits           0.06      2.9       0.14       6.7
!=                 0.06      2.9       0.06       2.9
.External          0.06      2.9       0.06       2.9

Once the slow parts identified the next step is to go back to the “drawing board” and start thinking about better ways of doing what you’re doing. From my personal experience here is a list of things to avoid when writing R code

• Loops: they are a real “plague” in R. Use lapply, sapply, tapply … functions instead whenever possible.
• Which statement: very often a good deal of time is spent on this statement when I can’t avoid using it
• datatype conversion: is very time consuming. Avoiding converting from one type to another and using only numeric make usually things running faster.

Finally Matthew Dowle has written an excellent package called data.table that allows “fast subset, fast grouping and fast merge in a short and flexible syntax”. The package provides a significant improvement in performance when working with large datasets.

Conclusion

The tools presented in this article are only a small sample of what is available in the R world and represent what I found to be the most useful. Obviously it depends largely on personal tastes and the kind of task at hand but from my experience it is a rather efficient set up.

I didn’t touch upon parallel computing which is high up on the list of priorities when dealing with large datasets. The keen reader can have a look at the large literature and the various packages on the topic (here)