Blog Archives

Estimate Age from First Name

July 29, 2013
By
Estimate Age from First Name

Today I read a cute post from Flowing Data on the most trendy names in US history. What caught my attention was a link posted in the article to the source data, which happens to be yearly lists of baby … Continue reading →

Read more »

save.ffdf and load.ffdf: Save and load your big data – quickly and neatly!

July 26, 2013
By
save.ffdf and load.ffdf: Save and load your big data – quickly and neatly!

I’m very indebted to the ff and ffbase packages in R.  Without them, I probably would have to use some less savoury stats program for my bigger data analysis projects that I do at work. Since I started using ff … Continue reading →

Read more »

Access individual elements of a row while using the apply function on your dataframe (or “applying down while thinking across”)

July 2, 2013
By
Access individual elements of a row while using the apply function on your dataframe (or “applying down while thinking across”)

The apply function in R is a huge work-horse for me across many projects.  My usage of it is pretty stereotypical.  Usually, I use it to make aggregations of a targeted group of columns for every row in a dataframe. … Continue reading →

Read more »

Which Torontonians Want a Casino? Survey Analysis Part 2

May 17, 2013
By
Which Torontonians Want a Casino?  Survey Analysis Part 2

In my last post I said that I would try to investigate the question of who actually does want a casino, and whether place of residence is a factor in where they want the casino to be built.  So, here … Continue reading →

Read more »

When the “reorder” function just isn’t good enough…

May 6, 2013
By
When the “reorder” function just isn’t good enough…

The reorder function, in R 3.0.0, is behaving strangely (or I’m really not understanding something).  Take the following simple data frame: df = data.frame(a1 = c(4,1,1,3,2,4,2), a2 = c(“h”,”j”,”j”,”e”,”c”,”h”,”c”)) I expect that if I call the reorder function on the … Continue reading →

Read more »

Do Torontonians Want a New Casino? Survey Analysis Part 1

May 2, 2013
By
Do Torontonians Want a New Casino?  Survey Analysis Part 1

Toronto City Council is in the midst of a very lengthy process of considering whether or not to allow the OLG to build of a new casino in Toronto, and where.  The process started in November of 2012, and set … Continue reading →

Read more »

Using ddply to select the first record of every group

April 13, 2013
By
Using ddply to select the first record of every group

I had a very long file of monetary transactions (about 207,000 rows) with about two handfuls of columns describing each transaction (including date).  The task I needed to perform on this file was to select the value from one of … Continue reading →

Read more »

Split, Apply, and Combine for ffdf

March 22, 2013
By
Split, Apply, and Combine for ffdf

Call me incompetent, but I just can’t get ffdfdply to work with my ffdf dataframes.  I’ve tried repeatedly and it just doesn’t seem to work!  I’ve seen numerous examples on stackoverflow, but maybe I’m applying them incorrectly.  Wanting to do some … Continue reading →

Read more »

Finding Patterns Amongst Binary Variables with the homals Package

February 10, 2013
By
Finding Patterns Amongst Binary Variables with the homals Package

It’s survey analysis season for me at work!  When analyzing survey data, the one kind of analysis I have realized that I’m not used to doing is finding patterns in binary data.  In other words, if I have a question … Continue reading →

Read more »

Multiple Classification and Authorship of the Hebrew Bible

January 1, 2013
By
Multiple Classification and Authorship of the Hebrew Bible

Sitting in my synagogue this past Saturday, I started thinking about the authorship analysis that I did using function word counts from texts authored by Shakespeare, Austen, etc.  I started to wonder if I could do something similar with the … Continue reading →

Read more »