PCA

PCA or Polluting your Clever Analysis

August 31, 2012 | Christoph Molnar

When I learned about principal component analysis (PCA), I thought it would be really useful in big data analysis, but that's not true if you want to do prediction. I tried PCA in my first competition at kaggle, but it delivered bad results. This post illustrates how PCA can pollute ...

[Read more...]

MAT8886 reducing dimension using factors

February 16, 2012 | arthur charpentier

First, let us recall a standard result from linear algebra: "real symmetric matrices are diagonalizable by orthogonal matrices". Thus, any variance-covariance matrix can be written since a variance-covariance matrix is also definite positive. In ... [Read more...]

Publishing in Veterinary Academic Journals

May 10, 2011 | denishaine

Following the post by Arthur Charpentier (Freakonometrics), I wondered what would be the outcome considering my current engagement (veterinary medicine, epidemiology, bovine mastitis). Briefly, Arthur Charpentier’s post looked at clusters of journals publishing the same kind of papers. So I looked at 25 journals (Journal of Dairy Science, Canadian Journal ... [Read more...]

Eigensheep

March 13, 2011 | Edwin Chen

Aaron Koblin’s Sheep Market visualization is an awesome use of Mechanical Turk. But it’d be even more awesome if the grid were ordered, so inspired by the use of eigenfaces in facial recognition, I decided to try projecting the sheep … Continue reading →

[Read more...]

Think academic journals look the same ? Well, some do…

February 8, 2011 | arthur charpentier

We have seen yesterday that finding an optimal strategy to publish is not that simple. And actually, it can be even more difficult in the case the journal rejects the paper (not because it is not correct, but because "it does not fit" with the stan... [Read more...]

Clustering NHL Skaters

February 6, 2011 | --

I have been sitting on this post for some time now and wanted to get it out there. The goal is to simply show how easy it is to pull live data from the web into R, massage it, and perform some analytics on it. I am not sure how ... [Read more...]

Generating stress scenarios: null correlation is not enough

December 28, 2010 | arthur charpentier

In a recent post (here, by @teramonagi), Teramonagi mentioned the use of PCA to model yield curve, i.e. to obtain the three factor, "parallel shift", "twist" and "butterfly". As in Nelson & Siegel, if m is maturity, is the yield of the cu...