Chop, Slice and Dice Your Returns in R

April 24, 2011
By
Chop, Slice and Dice Your Returns in R

I have a knife rack on my kitchen wall with all my kitchen knives easily identifiable and accessible. I also have small scars on my hand where each knife can claim to have left a mark. It's not the knife's fault, of course. They hardly like being sudde...

Read more »

RcppArmadillo 0.2.19

April 24, 2011
By

Last Monday, Conrad Sanderson released version 1.2.10 of his most excellent Armadillo templated C++ library for linear algebra; I followed up the same day with version 0.2.19 of our RcppArmadillo wrapper for R based on our Rcpp library. However, the...

Read more »

Logistic Regression & Factors in R

April 24, 2011
By
Logistic Regression & Factors in R

Factors are R's enumerated type. Suppose you define the variable cities -- a vector of strings -- whose possible values are "New York," "Paris," "London" and "Beijing." Instead of representing each city as a string of characters, you might prefer to ...

Read more »

Location Tracking on Android, too!

April 23, 2011
By
Location Tracking on Android, too!

This week it was revealed that the iPhone stores users’ locations, and this immediately caused a huge firestorm of commentary by tech geeks, panic among privacy advocates, and delight to data geeks like myself. Even better/worse, it seems that the iPhone caches location traces long-term, possibly back to the date the phone was activated. I ditched my iPhone this past...

Read more »

Dates in R and the First Day of the Month

April 23, 2011
By
Dates in R and the First Day of the Month

I spent some time this morning learning about how R thinks about dates in R. I found this website to be a useful guide.Imagine that your data are dates in a standard format and you want a vector o...

Read more »

Dates in R and the First Day of the Month

April 23, 2011
By
Dates in R and the First Day of the Month

I spent some time this morning learning about how R thinks about dates in R. I found this website to be a useful guide.Imagine that your data are dates in a standard format and you want a vector o...

Read more »

Measuring association using odds ratios

Measuring association using odds ratios

In my last two posts, I have used the UCI mushroom dataset to illustrate two things.  The first was the use of interestingness measures to characterize categorical variables, and the second was the use of binary confidence intervals...

Read more »

Another nice Rcpp example

April 23, 2011
By

While preparing my slides for the Rcpp workshop this Thursday, I had wondered about more nice examples motivating Rcpp. So I posed a quick question on the rcpp-devel list. And I received a few friendly answers. My favourite, so far, was a suggesti...

Read more »

Statisfaction on R-bloggers

April 23, 2011
By
Statisfaction on R-bloggers

This is the first post of Statisfaction on R-bloggers. As an introduction: we are PhD students and postdocs at CREST, a research centre on economics and statistics located in Paris, France. We jointly share tips and tricks useful in our everyday jobs, links to various pages, articles, conferences, seminars, including a PhD student seminar at

Read more »

Michael Ryder Streaks

April 22, 2011
By
Michael Ryder Streaks

A ways back I put up a post that uses R to plot the scoring trends of an NHL player. Given the recent chatter on sports talk radio around Boston, I used my script to plot the data for Michael … Continue reading →

Read more »

Intro

April 22, 2011
By
Intro

This blog will show you how to build tools to survive in the modern world. I will focus on statistics and machine learning, because that's where my strengths lie, but sometime we may find ourselves veering far off course.My primary interest lies in us...

Read more »

Zoo Slows Down Your Linear Model Function

April 22, 2011
By

I was a bit frustrated when I read Aris's comment to this post about speed of his calculations in Matlab. So I changed the time span of my dataset to 5 years and repeated the whole code. It was VERY disappointing to get the results after more than 5 ho...

Read more »

How to load your iPhone location data into R

April 22, 2011
By
How to load your iPhone location data into R

Earlier this week, data scientists Pete Warden and Alasdair Allen reported that iPhones and cell-enabled iPads keep an internal log of the devices location, which is accessible from the backup that iTunes creates when you sync the device. (Update Apr 27: Apple responds that the locations are those of nearby cell towers and wi-fi hotspots, not the device itself.)...

Read more »

Visualizing iPhone location tracking with R and Google Maps

April 22, 2011
By
Visualizing iPhone location tracking with R and Google Maps

Visualize location logs created by the iPhone using R and Google Maps.

Read more »

Big data problems

April 22, 2011
By
Big data problems

I have big data problems.I need to analyze 100s of millions of rows of data and tried hard for 2 weeks to see if I can use R for this. My assessment so far from the experiments...1) R is best for data that fits a computer's RAM (so get more RAM if...

Read more »

Big data problems

April 22, 2011
By
Big data problems

I have big data problems.I need to analyze 100s of millions of rows of data and tried hard for 2 weeks to see if I can use R for this. My assessment so far from the experiments...1) R is best for data that fits a computer's RAM (so get more RAM if...

Read more »

Statistical analyses using R

April 22, 2011
By
Statistical analyses using R

Another book I received from the Short Book Reviews section of the International Statistical Review is Everitt’s and Hothorn’s Handbook of statistical analyses using R. Here is a version of my book review. This book is the second (blue) edition of a successful (violet) handbook that can benefit a wide audience interested in using

Read more »

Sexy, Geeky Graphs using ggplot2 in R

April 22, 2011
By
Sexy, Geeky Graphs using ggplot2 in R

So I’ve been looking for some data to play with while learning R, other than the data I’m analysing for various experiments and papers I’m working on. I thought to myself, “Hey, this R stuff is pretty geeky. Can I engage in a higher level of geekiness?” And I think I’ve found a way: using

Read more »

How to Build a Dataset in R using an RSS feed or Web page

April 22, 2011
By
How to Build a Dataset in R using an RSS feed or Web page

I recently wanted to build a dataset from content in an RSS feed - the feed of crimes in Newark provided by SpotCrime.  (They have feeds for lots of US cities, but I just wanted Newark.  Please read their Terms of Service before using this co...

Read more »

How to Build a Dataset in R using an RSS feed or Web page

April 22, 2011
By
How to Build a Dataset in R using an RSS feed or Web page

I recently wanted to build a dataset from content in an RSS feed - the feed of crimes in Newark provided by SpotCrime.  (They have feeds for lots of US cities, but I just wanted Newark.  Please read their Terms of Service before using this co...

Read more »

Lack of confidence [revised]

April 21, 2011
By
Lack of confidence [revised]

Following the comments on our earlier submission to PNAS, we have written (and re-arXived) a revised version where we try to spell out (better) the distinction between ABC point (and confidence) estimation and ABC model choice, namely that the problem was at another level for Bayesian model choice (using posterior probabilities). When doing point estimation

Read more »

Great FAJ Article on Statistical Measure of Financial Turbulence

April 21, 2011
By
Great FAJ Article on Statistical Measure of Financial Turbulence

I particularly liked this well-written paper, since unlike most academic research, I was able to understand it, replicate it, and incorporate it.  I know that the Financial Analyst Journal is not considered by the academic community as a top-ti...

Read more »

stalkR: R functions for exploring iPhone and iPad (OS X only)

April 21, 2011
By
stalkR: R functions for exploring iPhone and iPad (OS X only)

Yesterday Alasdair Allan and Pete Warden shocked the world by revealing that iPhones and iPads have been keeping track of our every move, and saving the data in obfuscated back up files. As my friend Vince Buffalo mentioned on Twitter, part of me was disgusted by the secret stalking Steve Jobs was doing, but my

Read more »

Permutation Test with Stratified Data and Repeated Measurements

April 21, 2011
By
Permutation Test with Stratified Data and Repeated Measurements

This is an example for a permutation test on stratified samples with repeated measurements. Samples are interdependent firstly because they come from several sites and secondly because the sampling was repeated a second time. That is samples of the sam...

Read more »

Open Source replacements for Operations Research and Analytics Software

April 21, 2011
By
Open Source replacements for Operations Research and Analytics Software

I was reading an article from Datamation on 70 Open Source Replacements for Small Business when I noticed a glaring omission.  Where are the software applications for Operations Research and Analytics?  So here is my best addendum to this art...

Read more »

Open Source replacements for Operations Research and Analytics Software

April 21, 2011
By
Open Source replacements for Operations Research and Analytics Software

I was reading an article from Datamation on 70 Open Source Replacements for Small Business when I noticed a glaring omission.  Where are the software applications for Operations Research and Analytics?  So here is my best addendum to this art...

Read more »

Good Looking Maps in R

April 21, 2011
By
Good Looking Maps in R

I haven't yet tried this, but it looks like a nice package.In one recent project I needed to draw several maps and visualize different kinds of geographical data on it. I found the combination of R/ggplot/maps package extremely flexible and powerful,...

Read more »

Good Looking Maps in R

April 21, 2011
By
Good Looking Maps in R

I haven't yet tried this, but it looks like a nice package.In one recent project I needed to draw several maps and visualize different kinds of geographical data on it. I found the combination of R/ggplot/maps package extremely flexible and powerful,...

Read more »

Thomas Bayes, 250 years later

April 21, 2011
By
Thomas Bayes, 250 years later

A link on R-bloggers signaled a series of blogs and videos by IBM Netezza about Thomas Bayes and the consequences of his theorem. Which made me realise this was indeed the 250th anniversary of his death, and that maybe we (as a collective, incl. ISBA) should have done something on April 17th… Before the Revolution

Read more »