Another way for the database challenged (such as myself!) for merging two datasets that share at least one common column… This recipe using the cross-platform stats analysis package, R. I use R via the R-Studio client, which provides an IDE wrapper around the R environment. So for example, here’s ... [Read more...]

For our forthcoming book on machine learning for hackers, John Myles White and I will discuss clustering, and various methods for doing so. One common method for clustering observations [...] [Read more...]

Last week I attended the OU Statistics conference on Visualisation and Presentation in Statistics (VIPS) (notes: here and here) One of the things that struck me from conversations and some of the presentations was that statistics – and in particular public engagement around statistics – appears to be lagging science efforts in ... [Read more...]

Coercing variables of character and numeric type into a single dataframe yields all vectors to be defined as factors all [Read more...]

Gompertz Model Visualization # Gomperz growth function gomp [Read more...]

Yesterday Alasdair Allan and Pete Warden shocked the world by revealing that iPhones and iPads have been keeping track of our every move, and saving the data in obfuscated back up files. As my friend Vince Buffalo mentioned on Twitter, part of me was disgusted by the secret stalking Steve ... [Read more...]

This post describes some preliminary results from a compatibility study of the SAS sas7bdat file format. The most current results stored in a github repository here: sas7bdat The ultimate goal is a native solution to the incompatibility between open-source statistical software (e.g. R) and sas7bdat database ... [Read more...]

Look at just the first digit and the number of digits. science: 32914, 11566, 4989, 3743, 968, 814, 673, 482, 286, 2811 black and white: 1694, 1167, 1108, 988, 919, 639, 596, 591, 580, 544 lol: 22627, 18100, 17688, 14374, 13459, 12045, 4711, 3779, 36... [Read more...]

Once in a while, you embark on what looks like a simple computational procedure only to encounter frustration very early on. “I can’t even read my file into R!” you cry. Step back, take a deep breath and take note of what the software is trying to tell you. ... [Read more...]

Recently, the good folks at Infochimps.com rolled out a series of new APIs to add to their already impressive set of data resources. I have been in a perpetual state of catch-up since the new year, so I have only now got around to adding some of these new ... [Read more...]

So I have this strange date and time string, which I would like to convert to a “useable” date, i.e., something that a spreadsheet programme or R can work with. It looks like this (MON has 3 chars): ddMONyr:hh:mm:ss The … Continue reading → [Read more...]

[Read more...]

[Read more...]

From the HRSA website: [The ARF 2009] is a database containing more than 6,000 variables for each of the nation’s counties. ARF contains information on health facilities, health professions, measures of resource scarcity, health status, economic activity, health training programs, and socioeconomic and environmental characteristics. The data file itself is formatted ... [Read more...]

   I’ve been playing with Twitter data over the last week, archiving Algerian, Egyptian, Iranian, and Chinese tweets.  I thought I’d bring the story a little closer to home this time by archiving tweets from Wisconsin Union protesters on the … Continue reading → [Read more...]

 I’ve posted three examples of Twitter hashtags datasets in the last week: one on China, one on Iran, and one on Algeria.  In order to build these datasets, I needed to obtain older tweets; this is slightly more difficult than … Continue reading → [Read more...]

  Earlier this week, I posted a ~100k tweet dataset on the #25bahman protests in Iran.  The corresponding figure of frequencies showed a strong presence on Twitter, with over 500 tweets per 5 minute period at peak.  You can download the … Continue reading → [Read more...]

Because the next official release of Emacs will finally have a built-in package management system, I’ve been able to update the Emacs Starter Kit for the Social Sciences to make it easier to set up. AucTeX is now installed directly as a package, and so is ESS. While the ... [Read more...]

More starter kit stuff. Up till now, the Emacs Starter Kit for the Social Sciences included ESS, but bundled it with the git repo. A better option would be to have it installed via the package mechanism, like AucTeX is now, but it’s not included. The ELPA system is ... [Read more...]

New in nerdery this week, it’s now a bit easier to install the Emacs Starter Kit for the Social Sciences that I put together (based on lots of great work by Phil Hagelberg and, more recently, Eric Schulte). In the past, the fact that AucTeX was both necessary and ... [Read more...]