Merging Data Video Tutorial

April 25, 2011
By
Merging Data Video Tutorial

Here's a video tutorial where I walk through some code that does what the previous post describes.The FRED data is used extensively for macroeconomics. You might these data useful for joining in graph fights in the blogosphere.

Read more »

Merging Data Video Tutorial

April 25, 2011
By
Merging Data Video Tutorial

Here's a video tutorial where I walk through some code that does what the previous post describes.The FRED data is used extensively for macroeconomics. You might these data useful for joining in graph fights in the blogosphere.

Read more »

Further Adventures in Visualisation with ggplot2

April 25, 2011
By
Further Adventures in Visualisation with ggplot2

So I previously took a look at some data of player performance from a computer game. In this post, I’m going to do some further visualisations using ggplot2. The data consists of different types of player character, different roles for those characters, and their overall damage output (the unit here is damage per second, or

Read more »

RInside 0.2.4

April 25, 2011
By

After several months, it was time for a new release 0.2.4 of RInside which is now on CRAN. RInside is a set of convenience classes which facilitate embedding of R inside of C++ applications and programs, using the classes and functions provided by th...

Read more »

Merging Multiple Data Files into One Data Frame

April 24, 2011
By
Merging Multiple Data Files into One Data Frame

We often encounter situations where we have data in multiple files, at different frequencies and on different subsets of observations, but we would like to match them to one another as completely and systematically as possible. In R, the merge() comma...

Read more »

Merging Multiple Data Files into One Data Frame

April 24, 2011
By
Merging Multiple Data Files into One Data Frame

We often encounter situations where we have data in multiple files, at different frequencies and on different subsets of observations, but we would like to match them to one another as completely and systematically as possible. In R, the merge() comma...

Read more »

Of Height and Speed in Tennis, or Fuzziness and Techiness in College

April 24, 2011
By
Of Height and Speed in Tennis, or Fuzziness and Techiness in College

I thought of this after reading this post and perhaps also this one, one the Cheap Talk blog. Here's the puzzle: in general, being tall does not make you slow; but among professional tennis players, the tall athletes do tend to be relativel...

Read more »

Chop, Slice and Dice Your Returns in R

April 24, 2011
By
Chop, Slice and Dice Your Returns in R

I have a knife rack on my kitchen wall with all my kitchen knives easily identifiable and accessible. I also have small scars on my hand where each knife can claim to have left a mark. It's not the knife's fault, of course. They hardly like being sudde...

Read more »

RcppArmadillo 0.2.19

April 24, 2011
By

Last Monday, Conrad Sanderson released version 1.2.10 of his most excellent Armadillo templated C++ library for linear algebra; I followed up the same day with version 0.2.19 of our RcppArmadillo wrapper for R based on our Rcpp library. However, the...

Read more »

Logistic Regression & Factors in R

April 24, 2011
By
Logistic Regression & Factors in R

Factors are R's enumerated type. Suppose you define the variable cities -- a vector of strings -- whose possible values are "New York," "Paris," "London" and "Beijing." Instead of representing each city as a string of characters, you might prefer to ...

Read more »

Location Tracking on Android, too!

April 23, 2011
By
Location Tracking on Android, too!

This week it was revealed that the iPhone stores users’ locations, and this immediately caused a huge firestorm of commentary by tech geeks, panic among privacy advocates, and delight to data geeks like myself. Even better/worse, it seems that the iPhone caches location traces long-term, possibly back to the date the phone was activated. I ditched my iPhone this past...

Read more »

Dates in R and the First Day of the Month

April 23, 2011
By
Dates in R and the First Day of the Month

I spent some time this morning learning about how R thinks about dates in R. I found this website to be a useful guide.Imagine that your data are dates in a standard format and you want a vector o...

Read more »

Dates in R and the First Day of the Month

April 23, 2011
By
Dates in R and the First Day of the Month

I spent some time this morning learning about how R thinks about dates in R. I found this website to be a useful guide.Imagine that your data are dates in a standard format and you want a vector o...

Read more »

Measuring association using odds ratios

Measuring association using odds ratios

In my last two posts, I have used the UCI mushroom dataset to illustrate two things.  The first was the use of interestingness measures to characterize categorical variables, and the second was the use of binary confidence intervals...

Read more »

Another nice Rcpp example

April 23, 2011
By

While preparing my slides for the Rcpp workshop this Thursday, I had wondered about more nice examples motivating Rcpp. So I posed a quick question on the rcpp-devel list. And I received a few friendly answers. My favourite, so far, was a suggesti...

Read more »

Statisfaction on R-bloggers

April 23, 2011
By
Statisfaction on R-bloggers

This is the first post of Statisfaction on R-bloggers. As an introduction: we are PhD students and postdocs at CREST, a research centre on economics and statistics located in Paris, France. We jointly share tips and tricks useful in our everyday jobs, links to various pages, articles, conferences, seminars, including a PhD student seminar at

Read more »

Michael Ryder Streaks

April 22, 2011
By
Michael Ryder Streaks

A ways back I put up a post that uses R to plot the scoring trends of an NHL player. Given the recent chatter on sports talk radio around Boston, I used my script to plot the data for Michael … Continue reading →

Read more »

Intro

April 22, 2011
By
Intro

This blog will show you how to build tools to survive in the modern world. I will focus on statistics and machine learning, because that's where my strengths lie, but sometime we may find ourselves veering far off course.My primary interest lies in us...

Read more »

Zoo Slows Down Your Linear Model Function

April 22, 2011
By

I was a bit frustrated when I read Aris's comment to this post about speed of his calculations in Matlab. So I changed the time span of my dataset to 5 years and repeated the whole code. It was VERY disappointing to get the results after more than 5 ho...

Read more »

How to load your iPhone location data into R

April 22, 2011
By
How to load your iPhone location data into R

Earlier this week, data scientists Pete Warden and Alasdair Allen reported that iPhones and cell-enabled iPads keep an internal log of the devices location, which is accessible from the backup that iTunes creates when you sync the device. (Update Apr 27: Apple responds that the locations are those of nearby cell towers and wi-fi hotspots, not the device itself.)...

Read more »

Visualizing iPhone location tracking with R and Google Maps

April 22, 2011
By
Visualizing iPhone location tracking with R and Google Maps

Visualize location logs created by the iPhone using R and Google Maps.

Read more »

Big data problems

April 22, 2011
By
Big data problems

I have big data problems.I need to analyze 100s of millions of rows of data and tried hard for 2 weeks to see if I can use R for this. My assessment so far from the experiments...1) R is best for data that fits a computer's RAM (so get more RAM if...

Read more »

Big data problems

April 22, 2011
By
Big data problems

I have big data problems.I need to analyze 100s of millions of rows of data and tried hard for 2 weeks to see if I can use R for this. My assessment so far from the experiments...1) R is best for data that fits a computer's RAM (so get more RAM if...

Read more »

Statistical analyses using R

April 22, 2011
By
Statistical analyses using R

Another book I received from the Short Book Reviews section of the International Statistical Review is Everitt’s and Hothorn’s Handbook of statistical analyses using R. Here is a version of my book review. This book is the second (blue) edition of a successful (violet) handbook that can benefit a wide audience interested in using

Read more »

Sexy, Geeky Graphs using ggplot2 in R

April 22, 2011
By
Sexy, Geeky Graphs using ggplot2 in R

So I’ve been looking for some data to play with while learning R, other than the data I’m analysing for various experiments and papers I’m working on. I thought to myself, “Hey, this R stuff is pretty geeky. Can I engage in a higher level of geekiness?” And I think I’ve found a way: using

Read more »

How to Build a Dataset in R using an RSS feed or Web page

April 22, 2011
By
How to Build a Dataset in R using an RSS feed or Web page

I recently wanted to build a dataset from content in an RSS feed - the feed of crimes in Newark provided by SpotCrime.  (They have feeds for lots of US cities, but I just wanted Newark.  Please read their Terms of Service before using this co...

Read more »

How to Build a Dataset in R using an RSS feed or Web page

April 22, 2011
By
How to Build a Dataset in R using an RSS feed or Web page

I recently wanted to build a dataset from content in an RSS feed - the feed of crimes in Newark provided by SpotCrime.  (They have feeds for lots of US cities, but I just wanted Newark.  Please read their Terms of Service before using this co...

Read more »

Lack of confidence [revised]

April 21, 2011
By
Lack of confidence [revised]

Following the comments on our earlier submission to PNAS, we have written (and re-arXived) a revised version where we try to spell out (better) the distinction between ABC point (and confidence) estimation and ABC model choice, namely that the problem was at another level for Bayesian model choice (using posterior probabilities). When doing point estimation

Read more »

Great FAJ Article on Statistical Measure of Financial Turbulence

April 21, 2011
By
Great FAJ Article on Statistical Measure of Financial Turbulence

I particularly liked this well-written paper, since unlike most academic research, I was able to understand it, replicate it, and incorporate it.  I know that the Financial Analyst Journal is not considered by the academic community as a top-ti...

Read more »