I've finally had some success at munging some HTML. For quite some time, I've wanted to render a county level choropleth for US presidential election results. The numbers are all there on Politico.com, but attempts to use readHTMLTable never returned the full set of data. It still doesn't, but ... [Read more...]

For many of the people I talk to, Julia’s main appeal is speed. But achieving peak performance in Julia requires that programmers absorb a few subtle concepts that are generally unfamiliar to users of weakly typed languages. One particularly subtle performance pitfall is the need to write type-stable code. ... [Read more...]

A recent article by Matt Asay claims that "Python is displacing R as the language for data science". Python has certainly made some great strides in recent years, evolving beyond a data processing tool (an area where Python excels) to a data analysis tool. The Pandas project, in particular, has ... [Read more...]

As I go about cleaning and merging data sets with R I often end up creating and using simple functions over and over. When this happens, I stick them in the DataCombine package. This makes it easier for me to remember how to do an operation and others can possibly ... [Read more...]

I would like to recommend a recent workshop material on R/Bioconductor from Marc Carlson et al.http://www.bioconductor.org/help/course-materials/2013/SeattleMay2013/PDF: IntermediateSequenceAnalysis2013.pdfR script: IntermediateSequenceAnalysis201... [Read more...]

[Read more...]

This course is about getting to grips with the basics of the R command line for basic data analysis and visualisation. There's a focus on maps and ggplot2. The first part of the course uses Rich Harris's "Short Introduction to R" (here). The second part works through an "Introduction to ... [Read more...]

[Read more...]

It is a well known problem that in some models as the number of observations becomes large, econometric estimators fail to converge on consistent estimators.  The leading case of this is when estimating a binary response model with panel data with... [Read more...]

Some time back, I started a project on GitHub wherein I would explore the efficacy of financial literacy efforts in the area where I live. This is done with the support of a local non-profit organization. As a first step, I tried to draw a picture of the area at ... [Read more...]

With Stéphane Tufféry we’ve been working on credit scoring1 and we’ve been using the popular german credit dataset, __ myVariableNames credit = read.table( + "http://archive.ics.uci.edu/ml/machine-learning-databases/statlog/german/german.data", + header=FALSE,col.names=myVariableNames) __ credit$class library(RColorBrewer) __ CL=brewer.pal(6, "RdBu") __ ... [Read more...]

If you use attach, it is easy to tell if a variable exists. You can simply use exists to check:__attach(df)__exists("varName")[1] TRUEHowever, if you don't use attach (and I find you generally don't want to), this simple solution doesn't ... [Read more...]

I had the pleasure of visiting technology guru Robert Scoble at Rackspace Labs a few weeks ago. We had a great conversation about the Big Data revolution, and how large enterprises are catching up with the likes of Google and Facebook when it comes to using their data stores to ... [Read more...]

[Read more...]

Dear R-enthusiastic, I discovered Plotly some days ago, and I was fascinated by it. What is Plotly? Plotly is a service for creating and sharing data visualizations that also offers statistical analysis tools plus a robust API, the ability to … Continue reading → [Read more...]

I have been meaning to write a long post explaining each of the major R projects I am currently working on, but I can't seem to find the time to get to them. Instead, I give you a short summary of a few R projects/experiments/ideas I am kicking ... [Read more...]

So my first attempt to sort out the career of Michael Caine via parsing of HTML data was a wash. I'm going to try this again, using Wikipedia. They've got a nice, easy list of his films in an HTML table. Reading an HTML table into R is incredibly easy. ... [Read more...]

It is common for researchers to be concerned about unobserved effects being correlated with observed explanatory variables.For instance, if we were curious about the effect of meditation on emotional stability we may be concerned that there might be so... [Read more...]

Today I want to share another interesting idea contributed by Pierre Chretien. Pierre suggested using Averaged Input Assumptions and Momentum to create reasonably quiet strategy. The averaging techniques are used to avoid over-fitting any particular frequency. To create Averaged Input Assumptions we combine returns over different look-back periods, giving more ... [Read more...]

by Joseph Rickert R user groups are thriving. By our count of events listed on Revolution Analytics' Community Calendar, there were about 441 face-to-face R meetups worldwide in 2013; up 41% from 312 in 2012. The plot below indicates what the activity looks like on a monthly basis. (Download the data and R code behind ... [Read more...]