# Football, an ordinal model

September 30, 2012
By

(This article was first published on Wiekvoet, and kindly contributed to R-bloggers)

On September 19th, flo2speak remarked under a post that his/her experience is that ordinal models had better performance. That seems reason enough to try, so there we are. In examining this type of model it is found that more complex models can be used. There is now an interaction between defense powers and home/away and an effect of winter stop. This leads to a change in prediction values.

### Ordinal model and Log linear regression

#### ordinal model

An ordinal model has as a response a series of ordered categories. An example is liking of a food product (very much dislike, dislike, neutral, like, like very much) or stars for an Amazon product. Essential is that these are a fixed number of ordered categories. For football, the goals could be categories, although this does not have the fixed number of categories. In practice, this is not much of a problem, higher number of goals hardly occur, and the highest category is just including more goals up to infinity.
An ordinal model describes the chances for each of the categories. One might imagine this model as giving a continuous response, plus a mapping how this response is translated into the categories. So, if category A has been given limits l1 and l2, then the probability that a response falls between l1 and l2, is the probability of category A.
One of the nice properties of the ordinal model is that the categories don’t have to be the same size. It does not matter if the step from zero to one star is the same as the step from four to five stars. The model will adapt for that.

#### log linear model

The log linear model also predicts on a continuous scale. This scale is via the inverse-log (exponential) translated to a positive number. This number is then Poisson distributed. This means that a prediction has a confidence interval from the model, plus the variation due to the Poisson distribution. It should be noted that up to now my predictions were too precise, I only included the Poisson part. It also means that the predictions cannot become more ‘narrow’ for lack of a better word. It is not possible to get predictions which are saying there is 80% chance of one goal, because the Poisson distribution cannot make such outcomes. I can see that as a potential disadvantage.

### Application

#### Ordinal model in R

There are at least three packages with functions which can be used to fit ordinal models. The most obvious one is MASS which has the function polr. Almost as obvious is the package ordinal, which is a bit more extensive than just using polr. Finally, package VGAM has most capabilities. In this post I will do one demonstration of polr, and for the remainder use clm from package ordinal. VGAM is not used.
To demonstrate the equivalence between polr and clm:
top <- data.frame(OffenseClub=c(‘FC Groningen’,’Vitesse’)
(pol1 <- polr(oGoals ~OffenseClub,data=StartData))
Call:
polr(formula = oGoals ~ OffenseClub, data = StartData)

Coefficients:
OffenseClubAjax              OffenseClubAZ
2.16090602                 1.10515167
OffenseClubDe Graafschap       OffenseClubExcelsior
-0.07000361                -0.47817103
OffenseClubFC Groningen       OffenseClubFC Twente
0.05437062                 1.61304748
OffenseClubFC Utrecht       OffenseClubFeyenoord
0.72672725                 1.40465078
OffenseClubHeracles Almelo       OffenseClubNAC Breda
0.62502257                 0.38942582
OffenseClubNEC             OffenseClubPSV
0.26579358                 1.77961585
OffenseClubRKC Waalwijk         OffenseClubRoda JC
0.07824836                 0.63463829
OffenseClubSC Heerenveen         OffenseClubVitesse
1.64087778                 0.42797219
OffenseClubVVV-Venlo
0.14843577

Intercepts:
0|1        1|2        2|3        3|4        4|5        5|6        6|7
-0.5749746  0.8583642  1.9763217  2.9660666  4.1504225  4.9698549  6.2901004

Residual Deviance: 1929.761
AIC: 1977.761
(clm1 <- clm(oGoals ~OffenseClub ,data=StartData))
formula: oGoals ~ OffenseClub
data:    StartData

logit flexible  612  -964.88 1977.76 6(0)  6.04e-13

Coefficients:
OffenseClubAjax              OffenseClubAZ
2.16095                    1.10517
OffenseClubDe Graafschap       OffenseClubExcelsior
-0.07005                   -0.47813
OffenseClubFC Groningen       OffenseClubFC Twente
0.05446                    1.61290
OffenseClubFC Utrecht       OffenseClubFeyenoord
0.72677                    1.40463
OffenseClubHeracles Almelo       OffenseClubNAC Breda
0.62502                    0.38933
OffenseClubNEC             OffenseClubPSV
0.26583                    1.77970
OffenseClubRKC Waalwijk         OffenseClubRoda JC
0.07833                    0.63467
OffenseClubSC Heerenveen         OffenseClubVitesse
1.64085                    0.42778
OffenseClubVVV-Venlo
0.14849

Threshold coefficients:
0|1     1|2     2|3     3|4     4|5     5|6     6|7
-0.5750  0.8584  1.9763  2.9661  4.1504  4.9699  6.2902

predict(pol1,top,type=’p’)
0         1         2          3          4           5           6
1 0.3476590 0.3431691 0.1815278 0.07606575 0.03521247 0.009087138 0.005324424
2 0.2683624 0.3376048 0.2187080 0.10209439 0.04962636 0.013063007 0.007703923
7
1 0.001954373
2 0.002837110

predict(clm1,top,type=’p’)
\$fit
0         1         2          3          4           5           6
1 0.3476380 0.3431713 0.1815361 0.07607152 0.03521588 0.009087829 0.005325017
2 0.2684004 0.3376141 0.2186884 0.10207952 0.04961821 0.013060380 0.007702605
7
1 0.001954409
2 0.002836354

#### Selecting a start model

The feasible models are along the same lines as before. Offense, defense, home/away team, first/second half of the season. It is obvious that teams and hone/away are statistically significant.
clm0 <- clm(oGoals ~1,data=StartData)
clm2 <- clm(oGoals ~OffenseClub + DefenseClub,data=StartData)
clm3 <- clm(oGoals ~OffenseClub + DefenseClub +
OffThuis,data=StartData)
anova(clm0,clm1,clm2,clm3)
Likelihood ratio tests of cumulative link models:
clm0 oGoals ~ 1                                    logit flexible
clm1 oGoals ~ OffenseClub                          logit flexible
clm2 oGoals ~ OffenseClub + DefenseClub            logit flexible
clm3 oGoals ~ OffenseClub + DefenseClub + OffThuis logit flexible

no.par    AIC   logLik LR.stat df Pr(>Chisq)
clm0      7 2038.1 -1012.03
clm1     24 1977.8  -964.88  94.305 17  9.984e-13 ***
clm2     41 1951.1  -934.54  60.677 17  8.122e-07 ***
clm3     42 1922.3  -919.14  30.806  1  2.851e-08 ***
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ‘ 1
So, how does this model clm3 compare with model3? Using a slightly redrafted function fbpredict (see bottom of post) the prediction Roda JC vs FC Utrecht the individual outcomes are a bit different, but the summary is close enough not to make much differences. This is not so strange, in many ways the same model is build.
fbpredict(clm3,”Roda JC”,”FC Utrecht”)
\$details
Roda JC in rows against FC Utrecht in columns
0      1      2      3      4      5      6      7
0 0.0173 0.0424 0.0477 0.0297 0.0155 0.0040 0.0024 0.0009
1 0.0349 0.0856 0.0964 0.0599 0.0313 0.0082 0.0048 0.0018
2 0.0303 0.0741 0.0835 0.0519 0.0271 0.0071 0.0042 0.0015
3 0.0153 0.0375 0.0423 0.0263 0.0137 0.0036 0.0021 0.0008
4 0.0072 0.0176 0.0198 0.0123 0.0064 0.0017 0.0010 0.0004
5 0.0018 0.0044 0.0049 0.0031 0.0016 0.0004 0.0002 0.0001
6 0.0010 0.0026 0.0029 0.0018 0.0009 0.0002 0.0001 0.0001
7 0.0004 0.0009 0.0010 0.0006 0.0003 0.0001 0.0001 0

\$`summary chances`
Roda JC      equal FC Utrecht
0.4603544  0.2196707  0.3199749

#### Selecting model extensions; interactions

It is also possible to extend the model. This shows that there is an interaction between defense capabilities and playing home or away. The statistical significance is about p=0.06, which is low enough to consider this model for prediction purposes.

clm4a <- clm(oGoals ~OffenseClub*OffThuis + DefenseClub
,data=StartData)
clm4b <- clm(oGoals ~OffenseClub + DefenseClub*OffThuis
,data=StartData)
clm5 <- clm(oGoals ~(OffenseClub + DefenseClub)*OffThuis
,data=StartData)
anova (clm3,clm4a,clm5)
Likelihood ratio tests of cumulative link models:

clm3  oGoals ~ OffenseClub + DefenseClub + OffThuis   logit flexible
clm4a oGoals ~ OffenseClub * OffThuis + DefenseClub   logit flexible
clm5  oGoals ~ (OffenseClub + DefenseClub) * OffThuis logit flexible

no.par    AIC  logLik LR.stat df Pr(>Chisq)
clm3      42 1922.3 -919.14
clm4a     59 1938.2 -910.08  18.115 17    0.38164
clm5      76 1945.6 -896.81  26.552 17    0.06497 .

Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ‘ 1
anova (clm3,clm4b,clm5)
Likelihood ratio tests of cumulative link models:

clm3  oGoals ~ OffenseClub + DefenseClub + OffThuis   logit flexible
clm4b oGoals ~ OffenseClub + DefenseClub * OffThuis   logit flexible
clm5  oGoals ~ (OffenseClub + DefenseClub) * OffThuis logit flexible

no.par    AIC  logLik LR.stat df Pr(>Chisq)
clm3      42 1922.3 -919.14
clm4b     59 1930.7 -906.36  25.560 17    0.08286 .
clm5      76 1945.6 -896.81  19.107 17    0.32242

Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ‘ 1
The implication is that model ‘clm4b’ is now considered an alternative for model ‘clm3’. This makes for an interesting change in predictions. It can be seen that with this model FC Utrecht has a better chance to win than with ‘clm3’.

fbpredict(clm4b,”Roda JC”,”FC Utrecht”)
\$details
Roda JC in rows against FC Utrecht in columns
0      1      2      3      4      5      6      7
0 0.0379 0.0689 0.0503 0.0220 0.0093 0.0022 0.0012 0.0004
1 0.0701 0.1274 0.0931 0.0406 0.0172 0.0040 0.0023 0.0008
2 0.0523 0.0950 0.0694 0.0303 0.0128 0.0030 0.0017 0.0006
3 0.0231 0.0420 0.0307 0.0134 0.0057 0.0013 0.0007 0.0003
4 0.0098 0.0179 0.0131 0.0057 0.0024 0.0006 0.0003 0.0001
5 0.0023 0.0041 0.0030 0.0013 0.0006 0.0001 0.0001 0
6 0.0013 0.0024 0.0017 0.0008 0.0003 0.0001 0      0
7 0.0005 0.0008 0.0006 0.0003 0.0001 0      0      0

\$`summary chances`
Roda JC      equal FC Utrecht
0.3696674  0.2506325  0.3797000

#### Winter break

On top of that, the effect of before/after winter break can be examined. It would seem that winter break does have an effect. There is not much point in trying to make predictions using the model with winter break. If winter break has an effect, so does summer break. This does however, fit well with the remark of Huub on  a previous post ‘I tried to predict the results using only the data from the present year, and got four correct (FC Utrecht, FC Groningen, FC Twente, PSV). Maybe it is an idea to combine both the data of last year and of the current season and give a weight to the current year of, for example, 2. In this way there is still enough information in the model but it also accounts for more recent results…‘. Difference between years is a reason for more recent data to perform better.
StartData\$year <- factor(c(substr(old\$Datum,1,4),substr(old\$Datum,1,4)))
clm6 <- clm(oGoals ~OffenseClub + DefenseClub  + year + OffThuis
,data=StartData)
clm7 <- clm(oGoals ~(OffenseClub + DefenseClub)*year + OffThuis
,data=StartData)
anova (clm3,clm6,clm7)
Likelihood ratio tests of cumulative link models:
clm3 oGoals ~ OffenseClub + DefenseClub + OffThuis          logit flexible
clm6 oGoals ~ OffenseClub + DefenseClub + year + OffThuis   logit flexible
clm7 oGoals ~ (OffenseClub + DefenseClub) * year + OffThuis logit flexible

no.par    AIC  logLik LR.stat df Pr(>Chisq)
clm3     42 1922.3 -919.14
clm6     43 1924.2 -919.12  0.0341  1    0.85356
clm7     77 1945.7 -895.84 46.5557 34    0.07407 .
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ‘ 1
anova(clm3,clm7)

Likelihood ratio tests of cumulative link models:
clm3 oGoals ~ OffenseClub + DefenseClub + OffThuis          logit flexible
clm7 oGoals ~ (OffenseClub + DefenseClub) * year + OffThuis logit flexible
no.par    AIC  logLik LR.stat df Pr(>Chisq)
clm3     42 1922.3 -919.14
clm7     77 1945.7 -895.84   46.59 35    0.09106 .
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ‘ 1

#### Even more complex model

A final step is to combine the good of model ‘clm4b’ and ‘clm7’. It would seem these effects can very well be combined. This means that at some point a year effect must be included.

clmX <- clm(oGoals ~(OffenseClub + DefenseClub)*year + DefenseClub*OffThuis
,data=StartData)
anova(clm3,clmX)
Likelihood ratio tests of cumulative link models:

clm3 oGoals ~ OffenseClub + DefenseClub + OffThuis                        logit
clmX oGoals ~ (OffenseClub + DefenseClub) * year + DefenseClub * OffThuis logit
threshold:
clm3 flexible
clmX flexible

no.par    AIC  logLik LR.stat df Pr(>Chisq)
clm3     42 1922.3 -919.14
clmX     94 1951.3 -881.67  74.942 52     0.0203 *

Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ‘ 1
anova(clm4b,clmX)
Likelihood ratio tests of cumulative link models:

formula:
clm4b oGoals ~ OffenseClub + DefenseClub * OffThuis
clmX  oGoals ~ (OffenseClub + DefenseClub) * year + DefenseClub * OffThuis
clm4b logit flexible
clmX  logit flexible

no.par    AIC  logLik LR.stat df Pr(>Chisq)
clm4b     59 1930.7 -906.36
clmX      94 1951.3 -881.67  49.383 35    0.05424 .

Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ‘ 1
anova(clm7,clmX)
Likelihood ratio tests of cumulative link models:

clm7 oGoals ~ (OffenseClub + DefenseClub) * year + OffThuis               logit
clmX oGoals ~ (OffenseClub + DefenseClub) * year + DefenseClub * OffThuis logit
threshold:
clm7 flexible
clmX flexible

no.par    AIC  logLik LR.stat df Pr(>Chisq)
clm7     77 1945.7 -895.84
clmX     94 1951.3 -881.67  28.353 17    0.04098 *

Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ‘ 1

fbpredict <- function(object,club1,club2) {
UseMethod(‘fbpredict’,object)
}

fbpredict.polr <- function(object,club1,club2) {
top <- data.frame(OffenseClub=c(club1,club2),DefenseClub=c(club2,club1),OffThuis=c(1,0))
prepred <- predict(object,top,type=’p’)
oo <- outer(prepred[2,],prepred[1,])
rownames(oo) <- 0:(ncol(prepred)-1)
colnames(oo) <- rownames(oo)
class(oo) <- c(‘fboo’,class(oo))
attr(oo,’row’) <- club1
attr(oo,’col’) <- club2
wel <- c(sum(oo[upper.tri(oo)]),sum(diag(oo)),sum(oo[lower.tri(oo)]))
names(wel) <- c(club1,’equal’,club2)
return(list(details=oo,’summary chances’=wel))
}

fbpredict.clm <- function(object,club1,club2) {
top <- data.frame(OffenseClub=c(club1,club2),DefenseClub=c(club2,club1),OffThuis=c(1,0))
prepred <- predict(object,top,type=’p’)\$fit
oo <- outer(prepred[2,],prepred[1,])
rownames(oo) <- 0:(ncol(prepred)-1)
colnames(oo) <- rownames(oo)
class(oo) <- c(‘fboo’,class(oo))
attr(oo,’row’) <- club1
attr(oo,’col’) <- club2
wel <- c(sum(oo[upper.tri(oo)]),sum(diag(oo)),sum(oo[lower.tri(oo)]))
names(wel) <- c(club1,’equal’,club2)
return(list(details=oo,’summary chances’=wel))
}

print.fboo <- function(x,…) {
cat(attr(x,’row’),’in rows against’,attr(x,’col’),’in columns \n’)
class(x) <- class(x)[-1]
attr(x,’row’) <- NULL
attr(x,’col’) <- NULL
oo <- formatC(x,format=’f’,width=4)
oo <- gsub(‘\\.0+\$’,’       ‘,oo)
oo <- substr(oo,1,6)
print(oo,quote=FALSE,justify=’left’)
}

R-bloggers.com offers daily e-mail updates about R news and tutorials on topics such as: Data science, Big Data, R jobs, visualization (ggplot2, Boxplots, maps, animation), programming (RStudio, Sweave, LaTeX, SQL, Eclipse, git, hadoop, Web Scraping) statistics (regression, PCA, time series, trading) and more...