Life is short, use PythonI started to play with Python two weeks ago due to the limitation of R in terms of handling large data, then a friend of mine suggested me to try Python since I had to do data massage frequently, "Python is the best choice, trust me", ... [Read more...]

Just read a funny but much to the point blog entry on the difficulties of teaching proper programming skills to first year students! I will certainly make use of the style file as grading 180 exams is indeed a recurrent nightmare… Filed under: R,... [Read more...]

Recently I found myself needing to calculate the distance between a large number of longitude and latitude locations. As it turns out, because the earth is a three-dimensional object, you cannot simply pretend that you are in Flatland, albeit some … Continue reading → [Read more...]

Principal component analysis (PCA) is a mathematical transformation of possibly(correlated) variables into a number of uncorrelated variables called principal components. The resulting components from this transformation is defined in such a way that t... [Read more...]

Here are the slides from the first University of Utah and Research Park R Users Group meeting. They discuss getting help and finding packages. R [Read more...]

Nathan Yau has just published at FlowingData a step-by-step guide on making bubble charts in R. It's actually pretty simple: read in data, sqrt-transform the "bubble" variable (to scale the bubbles by area, not radius), and use the symbols function to plot. It's the last step, though, that really ups ... [Read more...]

R and the NYC R User Group get brief mentions in this article about AOL's offices in New York City. The NYC UseRs meet at AOL and (ironically) the next meeting on Dec 9 is on the topic of R at Google. New York Observer: Bringing Some Sizzle to the Dial-Up ... [Read more...]

Each year I have the pleasure (actually it’s quite fun) of teaching R programming to first year mathematics and statistics students. The vast majority of these students have no experience of programming, yet think they are good with computers because they use facebook! The class has around 100 students, and ... [Read more...]

In this post we will make use of football results data from the football-data.co.uk website to demonstrate creating functions in R to automate a series of standard operations that would be required for results data from various leagues and divisions. The first step is to consider what control ... [Read more...]

Matti Vihola has posted a new paper on arXiv about adaptive (random walk) Metropolis-Hastings algorithms. The update in the (lower diagonal) scale matrix is where is the current acceptance probability and the target acceptance rate; is the current random noise for the proposal, ; is a step size sequence decaying to ... [Read more...]

One of the current best tools in the machine learning toolbox is the 1930s statistical technique called logistic regression. We explain how to add professional quality logistic regression to your analytic repertoire and describe a bit beyond that. A statistical analyst working on data tends to deliberately start simple move ... [Read more...]

This post provides a brief introduction to make and makefiles. In particular it describes how to set up make on Windows with an emphasis on using make in Eclipse on projects involving R, Sweave, and LaTeX. Overview make is software that uses makefile... [Read more...]

As posted in early August from JSM 2010 in Vancouver, StatProb was launched as a way to promote an on-line encyclopedia/wiki with the scientific backup of expert reviewers. This was completely novel and I was quite excited to take part in the venture as a representative of the Royal Statistical ... [Read more...]

InfoChimps.com is mainly known as a clearinghouse for finding large data sets, for free or for sale. But they have also released (in beta, at least) an API that lets you find some pretty useful information on-demand. Normally, you'd have you use RESTful calls to access the API, but ... [Read more...]

[Read more...]

I recently got very close to the floating point trap, again, so here is a little tribute with some small examples! [Read more...]

One approach for analyzing RNASeq data from an organism with a well-annotated genome, is to align the reads to mRNA (cDNA) sequences instead of the genome. To do that you need to extract the transcript sequences from a database. This is how to extract ensembl transcript sequences from UCSC from ... [Read more...]

We know that the S&P 500 was up a little in 2007 and down a lot in 2008.  So on the surface the question seems really stupid.  But randomness played a part.  Let’s have a go at deciding how much of a part. Figure 1: Comparison of 2007 and 2008 for the S&P 500. ... [Read more...]

A new posting on arXiv by Madeleine Thompson on a graphical tool for assessing performance. She has developed a software called SamplerCompare, implemented in R and C. The graphical evaluation plots “log density evaluations per iteration times autocorrelation time against a tuning parameter in a grid of plots where rows ... [Read more...]

Yesterday I surfed the web looking for 3D wireframe examples to explain linear models in class. I stumbled across this site where animated 3D wireframe plots are outputted by SAS.  Below I did something similar in R. This post shows the few steps of how to create an animated .gif ... [Read more...]