Often times, a scatterplot reveals a pattern that seems not so linear. Polynomial regression can be used to explore a predictor at different levels of curvilinearity. This tutorial will demonstrate how polynomial regression can be used in a hierarchical fashion to best represent a dataset in R.
Tutorial FilesBefore we begin, you may want to download the sample data (.csv) used in this tutorial. Be sure to right-click and save the file to your R working directory. Note that all code samples in this tutorial assume that this data has already been read into an R variable and has been attached. This dataset contains hypothetical student data that uses practice exam scores to predict final exam scores.
The preceding scatterplot demonstrates that these data may not be linear. Notably, no one scored lower than 50 on the practice exam and at approximately the 85 and above practice mark, final exam scores taper off. These suggest that the data is curvilinear. Furthermore, since exam scores range between 0 to 100, it is not possible to observe nor appropriate to predict that an individual with a 150 practice score would have a certain final exam score.
Creating The Higher Order VariablesA two step process, identical to the one used to create interaction variables, can be followed to create higher order variables in R. First, the variables must be centered to mitigate multicollinearity. Second, the predictor must be multiplied by itself a certain number of times to create each higher order variable. In this tutorial, we will explore the a linear, quadratic, and cubic model. Therefore, the predictor will need to be squared to create the quadratic model and cubed to create the cubic model.
Step 1: CenteringTo center a variable, simply subtract its mean from each data point and save the result into a new R variable, as demonstrated below.
- > #center the independent variable
- > FinalC <- Final - mean(Final)
- > #center the predictor
- > PracticeC <- Practice - mean(Practice)
Step 2: MultiplicationOnce the input variable has been centered, the higher order terms can be created. Since a higher order variable is formed by the product of a predictor with itself, we can simply multiply our centered term from step one and save the result into a new R variable, as demonstrated below.
- > #create the quadratic variable
- > PracticeC2 <- PracticeC * PracticeC
- > #create the cubic variable
- > PracticeC3 <- PracticeC * PracticeC * PracticeC
Creating The ModelsNow we have all of the pieces necessary to assemble our linear and curvilinear models.
- > #create the models using lm(FORMULA, DATAVAR)
- > #linear model
- > linearModel <- lm(FinalC ~ PracticeC, datavar)
- > #quadratic model
- > quadraticModel <- lm(FinalC ~ PracticeC + PracticeC2, datavar)
- > #cubic model
- > cubicModel <- lm(FinalC ~ PracticeC + PracticeC2 + PracticeC3, datavar)
Evaluating The ModelsAs is the case in other forms of regression, it can be helpful to summarize and compare our potential models using the summary(MODEL) and anova(MODEL1, MODEL2,… MODELi) functions.
- > #display summary information about the models
- > summary(linearModel)
- > summary(quadraticModel)
- > summary(cubicModel)
- #compare the models using ANOVA
- anova(linearModel, quadraticModel, cubicModel)
At this point we can compare the models. In this case, the quadratic and cubic terms are not statistically significant themselves nor are their models statistically significant beyond the linear model. However, in a real research study, there would be other practical considerations to make before deciding on a final model.