In prior posts, I demonstrated how to download, calculate, and compare fantasy football projections from ESPN, CBS, and NFL.com.  In my last post, I demonstrated how to download FantasyPros projections, which aggregate projections from many different sources to increase prediction accuracy.  In this post, I will compare fantasy football projections from ESPN, CBS, NFL, and FantasyPros, including our average and latent projections to determine who has the best fantasy football projections.

The R Script

The R Script for downloading fantasy football projections from FantasyPros is located at:
https://github.com/dadrivr/FantasyFootballAnalyticsR/blob/master/R%20Scripts/Evaluate%20Projections.R

To compare the accuracy of the projections, we will use various metrics including:

• R-squared (R²) – higher is better
• Harrell’s c – higher is better
• Somer’s Dxy – higher is better
• Intraclass correlation (ICC) – higher is better
• Mean absolute error (MAE) – lower is better
• Root mean squared error (RMSE) – lower is better
• Mean absolute percentage error (MAPE) – lower is better
• Mean absolute scaled error (MASE) – lower is better

Whose Predictions Were the Best?

Note: MAPE was unable to be calculated for FantasyPros and the average because of values of zero in the series (for a discussion on this topic and for reasons to prefer MASE to the other error metrics, see here).

Here is how the projections ranked when focusing on R-squared and MASE:

1. FantasyPros
2. Latent
3. Average
4. CBS
5. ESPN
6. NFL.com

In general, projections from FantasyPros were more accurate than projections from ESPN, CBS, and NFL.com, and were also more accurate than our average and latent variables.  FantasyPros projections explained about 67% of the variance in the actual points scored in my Yahoo league in the 2012 season.  Interestingly, the average of the sources was more accurate than any of the individual sources.  Even better than the average was a latent variable representing the common variance of the sources, which discards the unique, error variance.

Here is a scatterplot of the FantasyPros projections in relation to the actual points scored:

Conclusion