On the growth of R and Python for data science
Want to share your content on R-bloggers? click here if you have a blog, or here if you don't.
A recent article by Matt Asay claims that “Python is displacing R as the language for data science”. Python has certainly made some great strides in recent years, evolving beyond a data processing tool (an area where Python excels) to a data analysis tool. The Pandas project, in particular, has greatly expanded Python's ability to handle statistical data sets (introducing an object akin to R's data frame), and added some time series handling tools. But Python is still a long, long way from being able to support the range of statistical procedures supported by the core R language, let alone those provided by the 5000 community-contributed packages in CRAN.
Asay's article is heavy on anecdote but light on actual data to support its claim. (ComputerWorld's Sharon Machlis does a great job pointing out the irony there.) Nonetheless, data do exist on R and Python usage; while there's no user-registration data for open-source projects, secondary sources can provide intelligence on how open source projects are being used. RStudio's Hadley Wickham uses data from the developer Q&A site StackOverflow to chart the number of open questions asked per month about R and Python, as a proxy for active usage:
As an general-purpose data processing tool, it's no surprise that Python has more activity than the domain-specific analytics language R. But it's clear that both are growing explosively (Wickham describes the growth as “very close to being exponential”). Looking closer though, we see that the proportion of R questions, as a fraction of Python questions, is also growing rapidly:
This belies the claim that Python is displacing R. In fact, this chart suggests the reverse is true, and that R usage is growing at a faster rate than Python.
More data points come from user surveys. In the 2013 KDNuggets poll of top languages for analytics, data mining and data science, R was the most-used software for the third year running (60.9%), with Python in second place (38.8%). More tellingly, R's usage grew almost four times faster than Python's in 2013 versus 2012 (8.4 percentage points for R, compared to 2.7 percentage points for Python).
It's a similar story on the community side. R has more than 125 active user groups worldwide, and the number of user group meetings has increased by 41% in the last year. Python has around 400 user groups (I couldn't find stats on the growth rate), but RedMonk's Stephen O'Grady compares the communities devoted to data science:
At RedMonk, we typically bet on the bigger community, but that’s not as easy here. Python’s total community is obviously much larger, but it seems probable that R’s community, which is more or less strictly focused on data science, is substantially larger than the subset of the Python community specifically focused on data.
My personal take is that there's more than enough room for both Python and R. As the data science boom continues, both will continue to grow as more and more practitioners enter the world of statistical computing. Some (especially those that come from a computer-science background) will choose Python. And those that come from a statistics or data science background will choose R (or will have already learned R in their studies). And even some that come from the die-hard developer community will end up loving R. But both communities will consider to advance the art of data science, and as open-source communities will inevitably cross-pollinate each other. R has already influenced Python in the realm of data analysis, and it would be no bad thing if Python were to influence R in other areas. That, after all, is the beauty of open source software.
R-bloggers.com offers daily e-mail updates about R news and tutorials about learning R and many other topics. Click here if you're looking to post or find an R/data-science job.
Want to share your content on R-bloggers? click here if you have a blog, or here if you don't.