Global energy forecasting competitions

February 19, 2014
By

(This article was first published on Hyndsight » R, and kindly contributed to R-bloggers)

The 2012 GEFcom competition was a great success with several new innovative forecasting methods introduced. These have been published in the IJF as follows:

These papers will appear in print in the next issue of the journal. The data and code for each method is also available.

GEFCom 2014

The 2014 edition of the competition is now under way with a probabilistic focus. The great thing about these competitions and the associated papers is that they provide a serious empirical comparison of the forecast capability of different algorithms. There are lots of suggested approaches to forecasting energy, but without head-to-head comparisons like this, it is not always clear what is a good method, and what methods can be discarded.

The developers of the best performing methods in the competition will be invited to submit a paper to a special issue of the IJF on probabilistic energy forecasting, to be edited by Tao Hong and Pierre Pinson.

In addition to the regular competition, there is also a competition for students where the best-performing classes can win a prize. See Tao Hong’s blog for further information.

Evaluating forecast distributions

Because this round of the competition is focusing on predicting the whole distribution, and not just a point forecast, we need a method of evaluating the forecasts which takes account of the whole distribution. We have chosen to use quantile scoring.

For each time period throughout the forecast horizon, the participants should provide the 0.01, 0.02, \dots, 0.99 quantiles — call these q_1,\dots,q_{99}, with q_0=-\infty or the natural lower bound, and q_{100}=\infty or the natural upper bound. The full predictive densities composed by these quantile forecasts are to be evaluated by the quantile score calculated through the pinball loss function.

For a quantile forecast q_a with a/100 as the target quantile, this score L is defined as:

    \[L(q_a, y) = \left\{ \begin{array}{ll} (1 - a/100) (q_a - y), & \text{if $y< q_a$};\\ a/100 (y - q_a), & \text{if $y\ge q_a$}; \end{array} \right.\]

where y is the observation used for verification, and a = 1, 2, \dots, 99.

To evaluate the full predictive densities, this score is then averaged over all target quantiles, from 0.01 to 0.99, for all time periods over all forecast horizons. The lower the score, the better the forecasts are.

Further information

For more information about the competition and how to enter, please visit www.gefcom.org.

To leave a comment for the author, please follow the link and comment on his blog: Hyndsight » R.

R-bloggers.com offers daily e-mail updates about R news and tutorials on topics such as: visualization (ggplot2, Boxplots, maps, animation), programming (RStudio, Sweave, LaTeX, SQL, Eclipse, git, hadoop, Web Scraping) statistics (regression, PCA, time series, trading) and more...



If you got this far, why not subscribe for updates from the site? Choose your flavor: e-mail, twitter, RSS, or facebook...

Comments are closed.