## How can you do a smart job getting data from internet?

October 9, 2011
By

I’d like to explore more the capabilities of my statistical packages to get data online and allocate it in memory instead of download each dataset by hand. After all, I found this task is pretty easy, but got me out of bed for one night trying to find the most efficient way to loop across

## The Skills of a Data Miner

October 9, 2011
By

Data mining is not only statistics, even if statistics is the most recognized academic component of it. It also includes data cleaning, machine learning and data visualization. The scarce factor is the ability to understand that data and extract value ...

## Equality of Covariances Matrices Test in R (varcomp)

October 9, 2011
By

This is a piece of code I implemented in 2004, which was supposed to be part of an R-package in multivariate testing (to be named, rather creatively, mvttests). Time has flown, I haven’t still got around to implementing the said package, but people keep asking me for the varcomp function, so here it is, for

## understanding computational Bayesian statistics

October 9, 2011
By

I have just finished reading this book by Bill Bolstad (University of Waikato, New Zealand) which a previous ‘Og post pointed out when it appeared, shortly after our Introducing Monte Carlo Methods with R. My family commented that the cover was nicer than those of my own books, which is true. Before I launch into

## Sphericity Test for Covariance Matrices in R (sphericity.test)

October 9, 2011
By

This is a piece of code I implemented in 2004, which was supposed to be part of an R-package in multivariate testing (to be named, rather creatively, mvttests). Time has flown, I haven’t still got around to implementing the said package, but people keep asking me for the sphericity.test function, so here it is, for

## Operating on datasets inside a function

October 9, 2011
By

There are times when we need to write a function that makes changes to a generic data frame that is passed as an argument. Let’s say, for example, that we want to write a function that converts to factor any … Continue reading →

## Open Street maps

October 8, 2011
By

There have been some exciting developments in the Deducer ecosystem over the summer which should go into CRAN release in the next few months. Today I'm going to give a quick sneak peek at an Open Street Map - R connection with accompanying GUI. This post will just show the non-GUI components. The first part of the

## Performance difference between Stata and R

October 8, 2011
By

With respect to multinomial logit model, the performance difference between the two packages are quite large, based on this post.

## Some light data munging with R, with an application to ranking NFL Teams

October 8, 2011
By

I recently submitted this blog to R-bloggers, which aggregates R-related blog posts. It's a fantastic site and has been invaluable to me as I've learned R. One of my favorite kinds of articles is the hands-on, "hello world"-style weekend project t...

## Visualizing GIS data with R and Open Street Map

October 8, 2011
By

In this post I way to share with you some code to use Openstreetmap – maps as a backdrop for a data visualization. We will use the RgoogleMaps-package for R. In the following I will show you how to make this graph. 1. Download the map I wanted to take a closer look at an

## A brief idea of style

October 8, 2011
By

Once one starts writing more R code the need for consistency increases, as it facilitates managing larger projects and their maintenance. There are several style guides or suggestions for R; for example, Andrew Gelman’s, Hadley Wickham’s, Bioconductor’s and this one. … Continue reading →

## Using Sweave

October 8, 2011
By

If you use R and haven’t discovered Sweave then go and find out about it. It enables R code and plots to be incorporated into a document so the analysis and report can be combined together in a single document. … Continue reading →

## R Graph Gallery widget in R Bloggers

October 8, 2011
By

The R Bloggers website, maintained by Tal Galili, aggregates blogs (including mine) from many people of the R community. Tal and I have been wondering about how to tight R Bloggers with the gallery, supporting each other's website. To that extent...

## Risk, Return and Analyst Ratings

October 7, 2011
By

Today I want to discuss a connection between Risk, Return and Analyst Ratings. Let’s start with defining our universe of stocks : 30 stocks from Dow Jones Industrial Average (^DJI) index. For each stock I will compute the number of Upgrades and Downgrades, Risk, and Return in 2010:2011. I will run a linear regression and

## Because it’s Friday: Reviews of Random Digits

October 7, 2011
By

If you dig around enough on Amazon.com, you can find some pretty odd products (like the Badonkadonk tank now sadly unavailable). Attached to these products you can often find a new form of comedy: the funny Amazon review. The products that attract such attention can be hard to fathom: this gallon of milk has more than 1,000 reviews. (Sample:...

## All combinations for levelplot

October 7, 2011
By

In a previous post I explained how to create all possible combinations of the levels of two factors using expand.grid(). Another use for this function is to create a regular grid for two variables to create a levelplot or a … Continue reading →

## In case you missed it: September Roundup

October 7, 2011
By

In case you missed them, here are some articles from September of particular interest to R users. The deadline to enter the "R Applications" contest with \$20,000 in prizes is October 31. The RHadoop Project, a new collection of open-source R packages from Revolution Analytics, makes it possible to write map-reduce jobs in R to analyze huge data sets...

## R Workshop: Reading in Large Data Frames

October 7, 2011
By

One question I get a lot about how to read large data frames into R. There are some useful tricks that can save you both time and memory when reading large data frames but I find that many people are not aware of them. Of course, your ability to read...

## When Wellington meets the “animation” package

October 7, 2011
By

The “animation” package is great for creating .gif files (of course, it also produces video and flash files thanks to Yihui Xie). By using this package, I would like to show you a nice spot in Wellington, NZ. At this … Continue reading →

## R – Tutorial I

October 7, 2011
By

NOTE : This tutorial has been superseded by the exhaustive R tutorials Here Basics Start R in Windows using the program menu. To quit :  q(). to call help for a function. help() or ?. use double quotes to escape special characters and tokens. e.g. ?”for”. objects() or ls() to obtain list of objects stored. rm()… Read More »

## FFT / Power Spectrum Box-and-Whisker Plot with Gggplot2

October 6, 2011
By

I have a bunch of time series whose power spectra (FFT via R's spectrum() function) I've been trying to visualize in an intuitive, aesthetically appealing way. At first, I just used lattice's bwplot, but the spacing of the X-axis here really matters. ...

## Visualizing Tables with plot.table

October 6, 2011
By

plot.table function in the Systematic Investor Toolbox is a flexible table drawing routine. plot.table has a simple interface and takes following parameters: plot.matrix – matrix with data you want to plot smain – text to draw in (top, left) cell; default value is blank string highlight – Either TRUE/FALSE to indicate if you want to

## Assumptions of the Linear Model

October 6, 2011
By

Linear Assumptions from the Analysis Factor – Assumptions of linear regression (and ANOVA) are about the residuals, not the normality or independence of the response variable (Y). If you don’t know what this means be sure to read this brief … Continue reading →

## Bat Country

October 6, 2011
By

I've spent a lot of time thinking about and using R's spectrum() function and the Fast Fourier Transform (FFT) in the last 5+ years. Lately, they've begun to remind me a little of a Theremin: simple to use, difficult to master. While prepping a figur...

## Webinar Oct 13: Successful uses of R in Banking

October 6, 2011
By

On Thursday October 13, Hong Ooi from ANZ (Australia and New Zealand Banking Group) will give a webinar presentation on Successful Uses of R (along with SAS and Excel) in Banking. We've covered Hong's use of R for credit risk analysis here on the blog before, and in next week's webinar he'll take an in-depth look at applying R...

## Efficient Frontier of Buy-Hold and Tactical System

October 6, 2011
By

In my mind, there are two very disparate views in the money management space: Markowitz style diversification and Faber style tactical allocation. I thought it would be fun to see what happens when we try to blend the two with an efficient frontier bet...

## Spatiotemporal Data Mining: 2

October 6, 2011
By

There are many visual methods used to identify patterns in space and time. I've discussed some in prior threads and will show a few others briefly here. One of the most difficult questions I often hear from others regarding markov type approaches, is...

## On R versus SAS

October 6, 2011
By

A short while ago there was a discussion on linkedin about the use of SAS versus R for the enterprise. I have thought a bit about the issue but, as I do not use Linkedin, I did not make any … Continue reading →

## A Work of Art: Efron on Bayesian Inference

October 6, 2011
By

(Contributing blogger Joseph Rickert reports from the Stanford University Statistics Seminar series - ed.) Stanford University is very gracious about letting the general public attend many university events. Yesterday, it caught my eye that Bradley Efron was going to speak on Bayesian inference and the parametric bootstrap at the weekly Statistics seminar. So, since the free shuttle that goes...