On Dataists, a new collaborative blog for data hackers that I’m contributing to, we’ve just announced a data contest that’s custom made for R users. To win the contest, you need to build a recommendation system for R packages. To find out more, check out the official ... [Read more...]

I’ve just released the newest version of ProjectTemplate. The primary change is a completely redesigned mechanism for automatically loading data. ProjectTemplate can now read compressed CSV files, access CSV data files over HTTP, read Stata, SPSS and RData binary files and even load MySQL database tables automatically. For my ... [Read more...]

[Read more...]

I’ve just released two new packages for R: log4r and SortableHTMLTables. log4r is a minimal logging utility for R that’s inspired by the log4j family of logging tools. It has substantially fewer features than other logging tools for R, but it’s hopefully easier to ... [Read more...]

Introduction Because R is, in part, a functional programming language, the ‘base’ package contains several higher order functions. By higher order functions, I mean functions that take another function as an argument and then do something with that function. If you want to know more about the usefulness of writing ... [Read more...]

Here’s a nice snippet from a 2009 article by Kass that I read yesterday: According to my understanding, laid out above, statistical pragmatism has two main features: it is eclectic and it emphasizes the assumptions that connect statistical models with observed data. The pragmatic view acknowledges that both sides of ... [Read more...]

We use frameworks like Ruby on Rails or ProjectTemplate to minimize the time we spend on irrelevant details. By definition, an irrelevant detail isn’t of interest to us. But how can we tell which details are irrelevant? This isn’t a trivial task and it seems to be, on ... [Read more...]

[Read more...]

Introduction I’ve just uploaded a new package to CRAN based on a stimulus generation algorithm that I use for my experiments on vision. The FourierDescriptors package provides methods for creating, manipulating and visualizing Fourier descriptors, which are a representational scheme used to describe closed planar contours. The canonical reference ... [Read more...]

This is a follow up to my recent post introducing the use of JAGS in R through the rjags package. In the comments on that post, Bernd Weiss encouraged me to write a short addendum that describes diagnostic functions that you should use to assess the output from an MCMC ... [Read more...]

I’m in the middle of a new project that involves analyzing the packages that are currently on CRAN. As part of my work, I could really benefit from information about which packages are installed on people’s computers. If you’re willing to part with a bit of your ... [Read more...]

Introduction As many people already know, I’ve recently uploaded a new R package called ProjectTemplate to GitHub and CRAN. The ProjectTemplate package provides a function, create.project(), that automatically builds a directory for a new R project with a clean sub-directory structure and automatic data and library loading tools. ... [Read more...]

Get Everything Set Up I’m going to assume that you have access to a machine that will run JAGS. If you don’t, then you should be able to use WinBUGS, which is very easy to get set up. Unfortunately, the details of what follows may not help you ... [Read more...]

Neil Kodner wrote a great post this morning about yesterday’s Twifficiency scores outbreak. He grabbed all the auto-tweeted scores he could find and plotted their distribution. I was struck by the asymmetry of the resulting distribution, which you can see below: Thankfully, Neil handed me the raw data for ... [Read more...]

Introduction This week I decided to start unit testing my R code, so I taught myself the bare minimum about the RUnit and testthat packages to be able to use them. Here’s what I found necessary to get started writing tests with both packages. RUnit Basic Example I’m ... [Read more...]

[Read more...]

In a world in which the price of calculation continues to decrease rapidly, but the price of theorem proving continues to hold steady or increase, elementary economics indicates that we ought to spend a larger and larger fraction of our time on calculation.1 Over the next ten years, I hope ... [Read more...]

Relevant to my earlier post about the relationship between direct democracy and laws prohibiting gay marriage, Pew Research just published poll data showing that a majority of Americans disapprove of same-sex marriage. [Read more...]

H. G. Wells famously said that, “statistical thinking will one day be as necessary for efficient citizenship as the ability to read and write.” I think we’re getting closer to that day: even the Supreme Court of the United States plan... [Read more...]

Given the recent votes on same-sex marriage in New Jersey and Portugal, I wanted to test a seemingly innocuous claim that touches upon very broad issues in political theory: does the degree of directness of a “democratic” vote predict whether the vote will promote or prohibit same-sex marriage? Naively, it ... [Read more...]