I browsed through UC Irvine Machine Learning Repository! the other day and noticed a nice data set regarding stone flakes produced by our ancestors, the prehistoric men. To quote the dataset owners:'The data set concerns the earliest history ... [Read more...]

While following up on Nina Zumel’s excellent Trimming the Fat from glm() Models in R I got to thinking about code style in R. And I realized: you can make your code much prettier by designing more of your functions to return data.frames. That may seem needlessly heavy-weight, ... [Read more...]

"A growing body of evidence that indicates that the most meaningful way to access predictive analytics and enhance the reputation of Data Science is through open source analytics, which greatly hinges upon the free open source programming language R", according to Dataversity in the recent article "The Relevance of Open ... [Read more...]

In a recent working paper from the prolific AQR Research Factory, the authors seek to dispel ten common myths of momentum investing.  To their credit, they use the fine data publicly available from Kenneth French and use fairly simple metrics to make ... [Read more...]

It was a great honour to be able to present to the China R Users Conference in Beijing last month, and share various applications of R that I've encountered recently. I was really invigorated by the enthusiasm of the many R users at the conference, and everyone was very warm ... [Read more...]

Interactive Parallel Coordinates with Multiple ColoursFor my research project, I need a tool to visualise results from multi-objective optimisations. Below is one of my early attempts using base R and parcoord in the MASS package, I have no problem using them for publication. However, these charts are all static. For ... [Read more...]

I have blogged a few days a ago about a study by Kiju Jung that suggested that implicit bias leads people to underestimate the danger of female-named hurricanes. The study used historical data to demonstrate a correlation between femininity and death-toll, and subsequent experiments seemed to show that people indeed ... [Read more...]

It’s something that I do surprisingly often: concatenating a list of data frames into a single (possibly quite enormous) data frame. Until now my naive solution worked pretty well. However, today I needed to deal with a list of over 6 million elements. The result was hours of page thrashing ... [Read more...]

This is a companion post to the article What’s the ROI of your loan? It depends…, which discusses approaches for …Continue reading → [Read more...]

I have been having a go in R at visualising player movements for the World Cup. I wanted to use similar plots to those used to visualise international migration flows in the recent Science paper that I co-authored. In the … Continue reading → [Read more...]

by Joseph Rickert Last week,I had the opportunity to participate in the Second Academy of Science and Engineering (ASE) Conference on Big Data Science and Computing at Stanford University. Since the conference was held simultaneously with the two other conferences, one on Social Computing and the other on Cyber ... [Read more...]

The usual route for importing data from spreadsheet applications like Excel or OpenOffice into R involves first exporting the data in CSV format. A newer (c. 2011) and more efficient CRAN package, called XLConnect, facilitates reading an entire Excel workbook and manipulating worksheets and cells programmatically from within R. XLConnect doesn't ... [Read more...]

[Read more...]

Andrew Tanentzap, William Lee, Adrian Monks, Kate Ladley, Peter Johnson, Geoffrey Rogers, Joy Comrie, Dean Clarke, and Ella Hayman write: We tested a multivariate hypothesis about the causal mechanisms underlying plant invasions in an ephemeral wetland in South Island, New Zealand to inform management of this biodiverse but globally imperilled ... [Read more...]

I'm the first (I hope) to admit when I've gotten something wrong. I like to think I'm humble enough to realize that there are limits to my knowledge. Actually, humility doesn't enter into it. Every day I'm confronted with things that I don't know or understand. Those same limits can ... [Read more...]

The str function is perhaps the most useful function in R. It provides great information about the structure of some object. When I teach R, especially for those coming from SPSS, the str function for data frames provides the information they are use to seeing on the variable view tab. ... [Read more...]

[Read more...]

A new minor / bug fix release 4.300.8 of Armadillo, the templated C++ library for linear algebra, was tagged by Conrad in his SVN repo a few days ago, following earlier snapshots in the 4.300.* series. We had prepared two earlier releases for GitHub... [Read more...]

In a recent issue of Significance, I discovered an interesting – and amuzing – figure, about some box & beard plot, in Dr Fisher’s casebook: Beard the statistician in his den. In French, the box plot (introduced by John Tukey, not George Box, as discussed in a previous post) is popular under ... [Read more...]

I wanted to write a quick post to introduce two repos that I created on Github that center on the NHL’s play-by-play data. 1.  My code repo that highlights how I crawl and store the raw JSON datasets can be found here 2.  I also created a very simple R Shiny ... [Read more...]