# General Linear Models The Basics

**Bluecology blog**, and kindly contributed to R-bloggers]. (You can report issue about the content on this page here)

Want to share your content on R-bloggers? click here if you have a blog, or here if you don't.

# General Linear Models: The Basics

General linear models are one of the most widely used statistical tool in the biological sciences. This may be because they are so flexible and they can address many different problems, that they provide useful outputs about statistical significance AND effect sizes, or just that they are easy to run in many common statistical packages.

The maths underlying General Linear Models (and Generalized linear models, which are a related but different class of model) may seem mysterious to many, but are actually pretty accessible. You would have learned the basics in high school maths.

We will cover some of those basics here.

## Linear equations

As the name suggests General Linear Models rely on a linear equation, which in its basic form is simply:

*y*_{i} = *α* + *β*x* _{*i*} + *ϵ*

_{i}

The equation for a straight line, with some error added on.

If you aren’t that familiar with mathematical notation, notice a few things about this equation (I have followed standard conventions here). I used normal characters for variables (i.e. things you measure) and Greek letters for parameters, which are estimated when you fit the model to the data.

*y*_{i} are your response data, I indexed the *y* with *i* to
indicate that there are multiple observations. *x*_{i} is
variously known as a covariate, predictor variable or explanatory
variable. *α* is an intercept that will be estimated. *α* has the same
units as y. (e.g. if y is number of animals, then *α* is expected the
number of animals when x = 0).

*β* is a slope parameter that will also be estimated. *β* is also termed
the effect size because it measures the effect of x on y. *β* has units
of ‘y per x’. For instance, if x is temperature, then *β* has units of
number of animals per degree C. *β* thus measures how much we expect y
to change if x were to increase by 1.

Finally, don’t forget *ϵ*_{i}, which is the error.
*ϵ*_{i} will measure the distance between each prediction of
*y*_{i} made by the model and the observed value of
*y*_{i}.

These predictions will simply be calculated as:

*y*_{i} = *α* + *β*x*_{i}

(notice I just removed the *ϵ*_{i} from the end). You can
think of the linear predictions as: the mean or ‘expected’ value a new
observation *y*_{i} would take if we only knew
*x*_{i} and also as the ‘line of best fit’.

## Simulating ideal data for a general linear model

Now we know the model, we can generate some idealized data. Hopefully this will then give you a feel for how we can fit a model to data. Open up R and we will create these parameters:

n <- 100 beta <- 2.2 alpha <- 30

Where `n`

is the sample size and `alpha`

and `beta`

are as above.

We also need some covariate data, we will just generate a sequence of
`n`

numbers from 0 to 1:

x <- seq(0, 1, length.out = n)

The model’s expectation is thus this straight line:

y_true <- beta * x + alpha plot(x, y_true)

Because we made the model up, we can say this is the true underlying relationship. Now we will add error to it and see if we can recover that relationship with a general linear model.

Let’s generate some error:

sigma <- 2.4 set.seed(42) error <- rnorm(n, sd = sigma) y_obs <- y_true + error plot(x, y_obs) lines(x, y_true)

Here `sigma`

is our standard deviation, which measures how much the
observations y vary around the true relationship. We then used `rnorm`

to generate `n`

random normal numbers, that we just add to our predicted
line `y_true`

to simulate observing this relationship.

Congratulations, you just created a (modelled) reality a simulated an ecologist going out and measuring that reality.

Note the `set.seed()`

command. This just ensures the random number
generator produces the same set of numbers every time it is run in R and
it is good practice to use it (so your code is repeatable). Here is a
great explanation of seed setting and why 42 is so
popular.

Also, check out the errors:

hist(error)

Looks like a normal distribution hey? That’s because we generated them from a normal distribution. That was a handy trick, because the basic linear model assumes the errors are normally distributed (but not necessarily the raw data).

Also note that `sigma`

is constant (e.g. it doesn’t get larger as x gets
larger). That is another assumption of basic linear models called
‘homogeneity of variance’.

## Fitting a model

To fit a basic linear model in R we can use the `lm()`

function:

m1 <- lm(y_obs ~ x)

It takes a formula argument, which simply says here that `y_obs`

depends
on (the tilde `~`

) `x`

. R will do all the number crunching to estimate
the parameters now.

To see what it came up with try:

coef(m1) ## (Intercept) x ## 30.163713 2.028646

This command tells us the estimate of the intercept (`(Intercept)`

) and
the slope on x under `x`

. Notice they are close to, but not exactly the
same as `alpha`

and `beta`

. So the model has done a pretty decent job of
recovering our original process. The reason the values are not identical
is that we simulated someone going and measuring the real process with
error (that was when we added the normal random numbers).

We can get slightly more details about the model fit like this:

summary(m1) ## ## Call: ## lm(formula = y_obs ~ x) ## ## Residuals: ## Min 1Q Median 3Q Max ## -7.2467 -1.5884 0.1942 1.5665 5.3433 ## ## Coefficients: ## Estimate Std. Error t value Pr(>|t|) ## (Intercept) 30.1637 0.4985 60.503 <2e-16 *** ## x 2.0286 0.8613 2.355 0.0205 * ## --- ## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1 ## ## Residual standard error: 2.511 on 98 degrees of freedom ## Multiple R-squared: 0.05357, Adjusted R-squared: 0.04391 ## F-statistic: 5.547 on 1 and 98 DF, p-value: 0.0205

I’m not going to go overboard with explaining this output now, but
notice a few key things. With the summary, we get standard errors for
the parameter estimates (which is a measure of how much they might
vary). Also notice the R-squared, which can be handy. Finally, notice
that the `Residual standard error`

is close to the value we used for
`sigma`

, which is because it is an estimate of `sigma`

from our
simulated data.

Your homework is play around with the model and sampling process. Try
change `alpha`

, `beta`

, `n`

and `sigma`

, then refit the model and see
what happens.

## Final few points

So did you do the homework? If you did, well done, you just performed a simple power analysis (in the broad sense).

In a more formal power analysis (which is what you might have come
across previously) could systematically vary `n`

or `beta`

and for 1000
randomised data sets and then calculate the proportion out of 1000
data-sets that your p-value was ‘significant’ (e.g. less than a critical
threshold like the ever-popular 0.05). This number tells you how good
you are at detecting ‘real’ effects.

Here’s a great intro to power analysis in the broad sense: Bolker, Ecological Models and Data in R

One more point. Remember we said above about some ‘assumptions’. Well we can check those in R quite easily:

plot(m1, 1)

This shows a plot of the residuals (A.K.A. errors) versus the predicted
values. We are looking for ‘heteroskedasticity’ which is a fancy way of
saying the errors aren’t equal across the range of predictions (remember
I said `sigma`

is a constant?).

Another good plot:

plot(m1, 2)

Here we are looking for deviations of the points from the line. Points on the line mean the errors are approximately normally distributed, which was a key assumption. Points far from the line could indicate the errors are skewed left or right, too fat in the middle, or too in the middle skinny. More on that issue here

## The end

So the basics might belie the true complexity of situations we can address with General Linear Models and their relatives Generalized Linear Models. But, just to get you excited, here are a few things you can do by adding on more terms to the right hand side of the linear equation:

- Model multiple, interacting covariates.
- Include factors as covariates (instead of continuous variables). Got a factor and a continuous variable? Don’t bother with the old-school ANCOVA method, just use a linear model.
- Include a spline to model non-linear effects (that’s a GAM).
- Account for hierarchies in your sampling, like transects sampled within sites (that’s a mixed effects model)
- Account for spatial or temporal dependencies.
- Model varying error variance (e.g. when the variance increases with the mean).

You can also change the left-hand side, so that it no longer assumes
normality (then that’s a **Generalized** Linear Model). Or even add
chains of models together to model pathways of cause and effect (that’s
a ‘path analysis’ or ‘structural equation model’)

If this taster has left you keen to learn more, then check out any one of the zillion online courses or books on GLMs with R, or if you can get to Brisbane, come to our next course (which as of writing was in Feb 2018, but we do them regularly).

Now you know the basics, practice, practice, practice and pretty soon you will be running General Linear Models behind your back while you watch your 2 year old, which is what I do for kicks.

**leave a comment**for the author, please follow the link and comment on their blog:

**Bluecology blog**.

R-bloggers.com offers

**daily e-mail updates**about R news and tutorials about learning R and many other topics. Click here if you're looking to post or find an R/data-science job.

Want to share your content on R-bloggers? click here if you have a blog, or here if you don't.