Posts Tagged ‘ prediction ’

Observing Dark Worlds – Visualizing dark matter’s distorting effect on galaxies

October 13, 2012
By
Observing Dark Worlds – Visualizing dark matter’s distorting effect on galaxies

Some people like to do crossword puzzles. I like to do machine learning puzzles. Lucky for me, a new contest was just posted yesterday on Kaggle. So naturally, my lazy Saturday was spent getting elbow deep into the data. The training set consists of a series of ‘skies’, each containing a bunch of galaxies. Normally,

Read more »

PCA or Polluting your Clever Analysis

August 31, 2012
By
PCA or Polluting your Clever Analysis

When I learned about principal component analysis (PCA), I thought it would be really useful in big data analysis, but that's not true if you want to do prediction. I tried PCA in my first competition at kaggle, but it delivered bad results. This post illustrates how PCA can pollute good predictors.When I started examining this problem,...

Read more »

Predictive analytics: Some ways to waste time

August 17, 2012
By
Predictive analytics: Some ways to waste time

I am starting to take part at different competitions at kaggle and crowdanalytics. The goal of most competitions is to predict a certain outcome given some covariables.  It is a lot of fun trying out different methods like random forests, boosted ...

Read more »

Experience on using R to build prediction models in business applications

March 8, 2012
By
Experience on using R to build prediction models in business applications

By Yanchang zhao, RDataMining.com Building prediction/classification models is one of the most widely-seen data mining tasks in business applications. To share experience on building prediction models with R, I have started a discussion at RDataMining group on LinkedIn with the … Continue reading →

Read more »

Prediction: the Lasso vs. just using the top 10 predictors

February 23, 2012
By
Prediction: the Lasso vs. just using the top 10 predictors

One incredibly popular tool for the analysis of high-dimensional data is the lasso. The lasso is commonly used in cases when you have many more predictors than independent samples (the n « p) problem. It is also often used in the context of predictio...

Read more »

ESPN Prediction Performance for the NFL

January 25, 2012
By
ESPN Prediction Performance for the NFL

Description:ESPN 'experts' predict the National Football League wins/losses each week.  The above chart shows the percentage of their correct guesses and an overall trend, week by week.Data:http://espn.go.com/nfl/picksAnalysis:The graph shows an i...

Read more »

Testing an S&P 500 prediction

July 10, 2011
By
Testing an S&P 500 prediction

If a particular prediction comes true, how surprised should we be? The prediction The page that sparked my curiosity tells of a prediction made a year ago that the S&P 500 would beat its historic high by the end of 2011.  It says that at the point the prediction was made, the level of the … Continue reading...

Read more »

Friday fun projects

May 14, 2011
By
Friday fun projects

What’s a “Friday fun project”? It’s a small computing project, perfect for a Friday afternoon, which serves the dual purpose of (1) keeping your programming/data analysis skills sharp and (2) providing a mental break from the grind of your day job. Ideally, the skills learned on the project are useful and transferable to your work

Read more »

Updating meteorological forecasts, part 1

November 7, 2010
By
Updating meteorological forecasts, part 1

As Mark Twain said "the art of prophecy is very difficult, especially about the future" (well, actually I am not sure Mark Twain was the  first one to say so, but if you're interested by that sentence, you can look here). I have been rather su...

Read more »

Baseball, basketball, and (not) getting better as time marches on

June 2, 2010
By
Baseball, basketball, and (not) getting better as time marches on

PROS ARE NOT GETTING BETTER AT FREE THROWS Rick Larrick recently told Decision Science News that baseball players have been getting better over the years in a couple ways. First, home runs and strikeouts have increased. The careless or clueless reader might note that this is curious, for from the batter’s perspective home runs are

Read more »