Some people like to do crossword puzzles. I like to do machine learning puzzles. Lucky for me, a new contest was just posted yesterday on Kaggle. So naturally, my lazy Saturday was spent getting elbow deep into the data. The training set consists of a series of ‘skies’, each containing a bunch of galaxies. Normally, 
When I learned about principal component analysis (PCA), I thought it would be really useful in big data analysis, but that's not true if you want to do prediction. I tried PCA in my first competition at kaggle, but it delivered bad results. This post illustrates how PCA can pollute good predictors.
When I started examining this problem,...
I am starting to take part at different competitions at kaggle and crowdanalytics. The goal of most competitions is to predict a certain outcome given some covariables. It is a lot of fun trying out different methods like random forests, boosted ...
By Yanchang zhao, RDataMining.com Building prediction/classification models is one of the most widely-seen data mining tasks in business applications. To share experience on building prediction models with R, I have started a discussion at RDataMining group on LinkedIn with the … Continue reading →
One incredibly popular tool for the analysis of high-dimensional data is the lasso. The lasso is commonly used in cases when you have many more predictors than independent samples (the n « p) problem. It is also often used in the context of predictio...
Description:ESPN 'experts' predict the National Football League wins/losses each week. The above chart shows the percentage of their correct guesses and an overall trend, week by week.Data:http://espn.go.com/nfl/picksAnalysis:The graph shows an i...
If a particular prediction comes true, how surprised should we be? The prediction The page that sparked my curiosity tells of a prediction made a year ago that the S&P 500 would beat its historic high by the end of 2011. It says that at the point the prediction was made, the level of the … Continue reading...
What’s a “Friday fun project”? It’s a small computing project, perfect for a Friday afternoon, which serves the dual purpose of (1) keeping your programming/data analysis skills sharp and (2) providing a mental break from the grind of your day job. Ideally, the skills learned on the project are useful and transferable to your work 
As Mark Twain said "the art of prophecy is very difficult, especially about the future" (well, actually I am not sure Mark Twain was the first one to say so, but if you're interested by that sentence, you can look here). I have been rather su...
PROS ARE NOT GETTING BETTER AT FREE THROWS Rick Larrick recently told Decision Science News that baseball players have been getting better over the years in a couple ways. First, home runs and strikeouts have increased. The careless or clueless reader might note that this is curious, for from the batter’s perspective home runs are
Zero Inflated Models and Generalized Linear Mixed Models with R.
Zuur, Saveliev, Ieno (2012).
