The Skills of a Data Miner

October 9, 2011
By

Data mining is not only statistics, even if statistics is the most recognized academic component of it. It also includes data cleaning, machine learning and data visualization. The scarce factor is the ability to understand that data and extract value ...

Read more »

Equality of Covariances Matrices Test in R (varcomp)

October 9, 2011
By
Equality of Covariances Matrices Test in R (varcomp)

This is a piece of code I implemented in 2004, which was supposed to be part of an R-package in multivariate testing (to be named, rather creatively, mvttests). Time has flown, I haven’t still got around to implementing the said package, but people keep asking me for the varcomp function, so here it is, for

Read more »

understanding computational Bayesian statistics

October 9, 2011
By
understanding computational Bayesian statistics

I have just finished reading this book by Bill Bolstad (University of Waikato, New Zealand) which a previous ‘Og post pointed out when it appeared, shortly after our Introducing Monte Carlo Methods with R. My family commented that the cover was nicer than those of my own books, which is true. Before I launch into

Read more »

Sphericity Test for Covariance Matrices in R (sphericity.test)

October 9, 2011
By
Sphericity Test for Covariance Matrices in R (sphericity.test)

This is a piece of code I implemented in 2004, which was supposed to be part of an R-package in multivariate testing (to be named, rather creatively, mvttests). Time has flown, I haven’t still got around to implementing the said package, but people keep asking me for the sphericity.test function, so here it is, for

Read more »

Operating on datasets inside a function

October 9, 2011
By

There are times when we need to write a function that makes changes to a generic data frame that is passed as an argument. Let’s say, for example, that we want to write a function that converts to factor any … Continue reading →

Read more »

Open Street maps

October 8, 2011
By
Open Street maps

There have been some exciting developments in the Deducer ecosystem over the summer which should go into CRAN release in the next few months. Today I'm going to give a quick sneak peek at an Open Street Map - R connection with accompanying GUI. This post will just show the non-GUI components. The first part of the

Read more »

Performance difference between Stata and R

October 8, 2011
By

With respect to multinomial logit model, the performance difference between the two packages are quite large, based on this post.

Read more »

Some light data munging with R, with an application to ranking NFL Teams

October 8, 2011
By
Some light data munging with R, with an application to ranking NFL Teams

I recently submitted this blog to R-bloggers, which aggregates R-related blog posts. It's a fantastic site and has been invaluable to me as I've learned R. One of my favorite kinds of articles is the hands-on, "hello world"-style weekend project t...

Read more »

Visualizing GIS data with R and Open Street Map

October 8, 2011
By
Visualizing GIS data with R and Open Street Map

In this post I way to share with you some code to use Openstreetmap – maps as a backdrop for a data visualization. We will use the RgoogleMaps-package for R. In the following I will show you how to make this graph. 1. Download the map I wanted to take a closer look at an

Read more »

A brief idea of style

October 8, 2011
By

Once one starts writing more R code the need for consistency increases, as it facilitates managing larger projects and their maintenance. There are several style guides or suggestions for R; for example, Andrew Gelman’s, Hadley Wickham’s, Bioconductor’s and this one. … Continue reading →

Read more »

Using Sweave

October 8, 2011
By

If you use R and haven’t discovered Sweave then go and find out about it. It enables R code and plots to be incorporated into a document so the analysis and report can be combined together in a single document. … Continue reading →

Read more »

R Graph Gallery widget in R Bloggers

October 8, 2011
By
R Graph Gallery widget in R Bloggers

The R Bloggers website, maintained by Tal Galili, aggregates blogs (including mine) from many people of the R community. Tal and I have been wondering about how to tight R Bloggers with the gallery, supporting each other's website. To that extent...

Read more »

Risk, Return and Analyst Ratings

October 7, 2011
By
Risk, Return and Analyst Ratings

Today I want to discuss a connection between Risk, Return and Analyst Ratings. Let’s start with defining our universe of stocks : 30 stocks from Dow Jones Industrial Average (^DJI) index. For each stock I will compute the number of Upgrades and Downgrades, Risk, and Return in 2010:2011. I will run a linear regression and

Read more »

Because it’s Friday: Reviews of Random Digits

October 7, 2011
By

If you dig around enough on Amazon.com, you can find some pretty odd products (like the Badonkadonk tank now sadly unavailable). Attached to these products you can often find a new form of comedy: the funny Amazon review. The products that attract such attention can be hard to fathom: this gallon of milk has more than 1,000 reviews. (Sample:...

Read more »

All combinations for levelplot

October 7, 2011
By
All combinations for levelplot

In a previous post I explained how to create all possible combinations of the levels of two factors using expand.grid(). Another use for this function is to create a regular grid for two variables to create a levelplot or a … Continue reading →

Read more »

In case you missed it: September Roundup

October 7, 2011
By

In case you missed them, here are some articles from September of particular interest to R users. The deadline to enter the "R Applications" contest with $20,000 in prizes is October 31. The RHadoop Project, a new collection of open-source R packages from Revolution Analytics, makes it possible to write map-reduce jobs in R to analyze huge data sets...

Read more »

R Workshop: Reading in Large Data Frames

October 7, 2011
By

 One question I get a lot about how to read large data frames into R. There are some useful tricks that can save you both time and memory when reading large data frames but I find that many people are not aware of them. Of course, your ability to read...

Read more »

When Wellington meets the “animation” package

October 7, 2011
By
When Wellington meets the “animation” package

The “animation” package is great for creating .gif files (of course, it also produces video and flash files thanks to Yihui Xie). By using this package, I would like to show you a nice spot in Wellington, NZ. At this … Continue reading →

Read more »

FFT / Power Spectrum Box-and-Whisker Plot with Gggplot2

October 6, 2011
By

I have a bunch of time series whose power spectra (FFT via R's spectrum() function) I've been trying to visualize in an intuitive, aesthetically appealing way. At first, I just used lattice's bwplot, but the spacing of the X-axis here really matters. ...

Read more »

Visualizing Tables with plot.table

October 6, 2011
By
Visualizing Tables with plot.table

plot.table function in the Systematic Investor Toolbox is a flexible table drawing routine. plot.table has a simple interface and takes following parameters: plot.matrix – matrix with data you want to plot smain – text to draw in (top, left) cell; default value is blank string highlight – Either TRUE/FALSE to indicate if you want to

Read more »

Assumptions of the Linear Model

October 6, 2011
By
Assumptions of the Linear Model

Linear Assumptions from the Analysis Factor – Assumptions of linear regression (and ANOVA) are about the residuals, not the normality or independence of the response variable (Y). If you don’t know what this means be sure to read this brief … Continue reading →

Read more »

Bat Country

October 6, 2011
By
Bat Country

I've spent a lot of time thinking about and using R's spectrum() function and the Fast Fourier Transform (FFT) in the last 5+ years. Lately, they've begun to remind me a little of a Theremin: simple to use, difficult to master. While prepping a figur...

Read more »

Webinar Oct 13: Successful uses of R in Banking

October 6, 2011
By

On Thursday October 13, Hong Ooi from ANZ (Australia and New Zealand Banking Group) will give a webinar presentation on Successful Uses of R (along with SAS and Excel) in Banking. We've covered Hong's use of R for credit risk analysis here on the blog before, and in next week's webinar he'll take an in-depth look at applying R...

Read more »

Efficient Frontier of Buy-Hold and Tactical System

October 6, 2011
By
Efficient Frontier of Buy-Hold and Tactical System

In my mind, there are two very disparate views in the money management space: Markowitz style diversification and Faber style tactical allocation. I thought it would be fun to see what happens when we try to blend the two with an efficient frontier bet...

Read more »

Spatiotemporal Data Mining: 2

October 6, 2011
By
Spatiotemporal Data Mining: 2

There are many visual methods used to identify patterns in space and time. I've discussed some in prior threads and will show a few others briefly here. One of the most difficult questions I often hear from others regarding markov type approaches, is...

Read more »

On R versus SAS

October 6, 2011
By

A short while ago there was a discussion on linkedin about the use of SAS versus R for the enterprise. I have thought a bit about the issue but, as I do not use Linkedin, I did not make any … Continue reading →

Read more »

A Work of Art: Efron on Bayesian Inference

October 6, 2011
By

(Contributing blogger Joseph Rickert reports from the Stanford University Statistics Seminar series - ed.) Stanford University is very gracious about letting the general public attend many university events. Yesterday, it caught my eye that Bradley Efron was going to speak on Bayesian inference and the parametric bootstrap at the weekly Statistics seminar. So, since the free shuttle that goes...

Read more »

R talk on regular expressions (regex)

October 6, 2011
By
R talk on regular expressions (regex)

Regular expressions are a powerful in any language to manipulate, search, etc. data. For example:> fruit <- c("apple", "banana", "pear", "pineapple")> fruit "apple" "banana" "pear" "pineapple"> grep("a", fruit) # there is an ...

Read more »

R: Preparing balanced stimuli lists for a psychological experiment

October 6, 2011
By

Dividing a list of stimuli described by several statistics into subsets which are balanced according to these statistics is a common task in psychological research. For the purpose of preparing materials for an experiment which I am going to conduct &#...

Read more »