Programmers have long been very proud and loyal with their tools, and often very vocal. This has led to well-contested rivalries and “fights” about which tool is better:

• emacs or vi;
• Java or C++;
• Perl or Python;
• Django or Rails;
• and, for data geeks, the SAS/SPSS/R/Matlab fight.

The truth is, very few of us data geeks (data scientists, data analysts, statisticians, or what ever we call ourselves [editor note: Data Practitioners]) use only a single tool for all of our work. We will often extract data from a SQL database, munge it using Perl or Python, and then do statistical analysis using R or SAS, reporting the results using Word or, increasingly, the web. Specially for data analysis, there is often no single tool that can do the end-to-end workflow well, however much we would like to believe that there is. Each tool has its strengths and weaknesses, and often a mixture works best. The trick is in finding the right “glue” that can string our workflow together.

There are now several interface packages available to talk between open-source languages. I’ll speak to the interfaces with R, which I’m most familiar with, but I’m sure that the community will point out other useful interfaces. R is not the fastest nor most elegant of languages, but has by far the richest ecosystem of cutting-edge data analysis packages. There are now ways to communicate with R from other general programming languages like Java (through the rJava package and JNI), Perl (Statistics::R, available in CPAN),  Python (rpy2, PypeR, available in PyPI). Packages in R allow communication out with general packages, like RSPython, RSPerl (both available at Omegahat) and rJava. Most commercial statistical packages, like SAS, SPSS and Statistica allow you to write R code to send to R and then get back the results. A specially nice SAS macro to do this for those without the latest versions of SAS is %Proc_R, available here. One can also call R from Matlab. There are also many ways of interfacing with R using web-based tools like Rserve or, on Windows, the rcom interface to utilize COM and connect with, among other things, Word and Excel.

More recently I have been excited about platforms where code can be written in different languages and integrated using literate programming (i.e., the weaving of the results of code with text to create reports).

• Babel is a part of org-mode in Emacs which allows different programming languages to be used in the same document to perform an analysis and report. There are several examples of how this is done.
• The latest IPython distribution now allows you to integrate other languages using user-contributed magic functions. The initial languages available are R, Octave and, very recently, Julia. The first two are already integrated into IPython. Using these magic functions, you can use the power of R, Octave and Julia along with all the tools available in Python like Numpy, Scipy, matplotlib, pandas and the like on one platform. Literate programming is easily achieved through the excellent HTML notebook that is now part of IPython distributions. Update: A sql magic function was just added to the ecosystem.

The interfacing tools I’ve described now allows us to create a greater ecosystem where different tools can be integrated to a common goal rather easily. Instead of fighting over which tool is better, we’re now going to a place where that doesn’t matter; what matters is being able to use the right tools for each piece of the job and getting the tools working together to do the best job possible. We can, after all, all get along.

PS: For translating code between Matlab/Octave, Python and R, there is a great little site called Mathesaurus.

(Note, DataCommunityDC is an Amazon Affiliate. Thus, if you click the image in the post and buy the book, we will make approximately $0.43 and retire to a small island).