Are Subscription Sources More Accurate?

[This article was first published on R – Fantasy Football Analytics, and kindly contributed to R-bloggers]. (You can report issue about the content on this page here)
Want to share your content on R-bloggers? click here if you have a blog, or here if you don't.

Introduction

In this article, we examine whether there are advantages to paying for subscription fantasy football projections. We tested whether projections from subscription sources have higher accuracy than projections from free, publicly available sources. There are arguments that subscription sources would possibly be more accurate as you may expect to get better accuracy as part of what you are paying for. We examined projections from the 2015 season for QB, RB, WR and TE positions.

Overall accuracy

We calculated the projected seasonal points based on standard scoring settings as used in our Projections tool and compared with the actual points. For the aggregation of sources we used the regular mean. We have 10 free sources and 6 subscription sources. The free sources were: CBS, Yahoo, ESPN, FOX, NFL, FFToday, NumberFire, EDS Football, WalterFootball and RTSports. Because the subscription sources are not publicly available we chose not to disclose the names of the sources. For each of the groups we calculated R2 (higher is better) and MASE (lower is better) values as well as values for both groups combined. The results are below.

Overall accuracy
Source Type R-Squared MASE
Free .635 .548
Subscription .618 .568
All .641 .541

Based on the results, the subscription sources were less accurate than the free sources but add to the overall accuracy. In light of that, one possible reason the subscription sources were less accurate could be because there were more free sources than subscription sources. To investigate that possibility, we examined the accuracy of all possible combinations of 6 sources among the free sources. The results below show the mean R2 and MASE values for all the combinations. Reducing the number of free sources did reduce the projections’ accuracy, but the free sources were still more accurate than the subscription sources. In other words, even after accounting for how many sources of projections were included, free projections were still more accurate than subscription projections.

All Possible Combinations of 6 Free Sources
R-Squared MASE
Combinations of free sources .631 .556

Position Accuracy

Let’s examine whether the results are different when we look at individual positions:

QB Accuracy
Source Type R-Squared MASE
Free .707 .416
Subscription .677 .434
All .705 .414
RB Accuracy
Source Type R-Squared MASE
Free .488 .673
Subscription .492 .686
All .503 .657
WR Accuracy
Source Type R-Squared MASE
Free .613 .575
Subscription .560 .633
All .616 .571
TE Accuracy
Source Type R-Squared MASE
Free .550 .583
Subscription .541 .592
All .559 .572

For the QB, WR, and TE positions, the free sources were more accurate than the subscription sources, while the subscription sources were slightly more accurate for the RB position measured by R2 but slightly less accurate as measured by MASE. For every position except the QB position, combining the free and subscription sources also increased the overall accuracy.

We also calculated all possible combinations of 6 sources among the free sources by position. As the results below show, the accuracy for the free sources did decrease.  However, as was the case with overall accuracy, the free sources were still more accurate than the subscription sources even after accounting for how many sources of projections were included.

All Possible Combinations of 6 Free Sources By Position
Position R-Squared MASE
QB .702 .422
RB .482 .688
WR .605 .585
TE .548 .589

Conclusion

We have seen that subscription sources are not more accurate than the free sources. In general, free projections were actually more accurate than subscription projections.  However, including subscription projections did improve the accuracy of projections both overall and for each position (except quarterbacks). As we have demonstrated before, individual analysts do not reliably beat the “Wisdom of the Crowd” and this analysis further supports that. So while free sources seem to be more accurate, on average, than subscription sources, it is combining them that adds to the accuracy of the overall projections. The most accurate projections combined free and subscriptions sources.  So if you are asking whether you should use free or subscription sources for your projections, the answer is: use both!

You can find data and script for the analysis here.

The post Are Subscription Sources More Accurate? appeared first on Fantasy Football Analytics.

To leave a comment for the author, please follow the link and comment on their blog: R – Fantasy Football Analytics.

R-bloggers.com offers daily e-mail updates about R news and tutorials about learning R and many other topics. Click here if you're looking to post or find an R/data-science job.
Want to share your content on R-bloggers? click here if you have a blog, or here if you don't.

Never miss an update!
Subscribe to R-bloggers to receive
e-mails with the latest R posts.
(You will not see this message again.)

Click here to close (This popup will not appear again)