Want to share your content on R-bloggers? click here if you have a blog, or here if you don't.
olsrr 0.7.1 is on CRAN. olsrr provides tools for building and checking ordinary least squares regression models in R: regression summaries, variable selection, heteroskedasticity and normality tests, collinearity diagnostics, influence measures and residual plots, all built around lm() objects.
We don’t usually write about patch releases, but this one deserves a post:
- Several test and summary results change: Holm-adjusted Breusch-Pagan
-values, the Cramér–von Mises normality test, and the residual MSE/RMSE in model summaries. If you report these numbers, please re-run your analyses. - Every calculation and plot was checked against base R and reference packages (
car,lmtestandnortest) for this release. - It is the first
olsrrpost since our introduction in 2019, so it also covers what has been added since then.
Who should read what: if you report Holm-adjusted Breusch-Pagan olsrr, read Behaviour changes first. Everyone else can skip to What’s new since 2019.
You can install it from CRAN with:
install.packages("olsrr")
You can see a full list of changes in the release notes.
< section id="behaviour-changes" class="level2">Behaviour changes
These fixes change numbers you may already have in reports. In each case 0.7.1 now agrees with the base R or reference implementation, so the new values are the ones to use.
How we checked: every number in this post was re-run on CRAN olsrr 0.7.0 and 0.7.1, and the release passed an independent audit against base R, car, lmtest and nortest before it was submitted.
Breusch-Pagan test: Holm-adjusted p-values
With ols_test_breusch_pagan(..., rhs = TRUE, multiple = TRUE, p.adj = "holm"), version 0.7.0 sorted the raw stats::p.adjust(p, "holm"). In this three-predictor example the raw 0.0465, 1.07e-05 and 0.0410:
set.seed(15) n <- 60 z <- rnorm(n) d <- data.frame(x1 = z + rnorm(n), x2 = z + rnorm(n), x3 = z + rnorm(n)) d$y <- rnorm(n) * exp(0.55 * z) m_bp <- lm(y ~ x1 + x2 + x3, data = d) ols_test_breusch_pagan(m_bp, rhs = TRUE, multiple = TRUE, p.adj = "holm")
Breusch Pagan Test for Heteroskedasticity
-----------------------------------------
Ho: the variance is constant
Ha: the variance is not constant
Data
-------------------
Response : y
Variables: x1 x2 x3
Test Summary (Holm's p values)
-------------------------------------------------
Variable chi2 df p
-------------------------------------------------
x1 3.962893 1 8.192284e-02
x2 19.381555 1 3.211077e-05
x3 4.177610 1 8.192284e-02
-------------------------------------------------
simultaneous 21.479644 3 8.369002e-05
-------------------------------------------------
olsrr 0.7.0
| x1 | 0.0465 |
| x2 | 3.21e-05 |
| x3 | 0.0819 |
olsrr 0.7.1
| x1 | 0.0819 |
| x2 | 3.21e-05 |
| x3 | 0.0819 |
Same call on both CRAN versions: ols_test_breusch_pagan(m_bp, rhs = TRUE, multiple = TRUE, p.adj = "holm"). Changed values are highlighted.
# Run once with each version, e.g. in two sessions:
# remotes::install_version("olsrr", "0.7.0") # then
# install.packages("olsrr") # 0.7.1
library(olsrr)
set.seed(15)
n <- 60
z <- rnorm(n)
d <- data.frame(x1 = z + rnorm(n), x2 = z + rnorm(n), x3 = z + rnorm(n))
d$y <- rnorm(n) * exp(0.55 * z)
m_bp <- lm(y ~ x1 + x2 + x3, data = d)
ols_test_breusch_pagan(m_bp, rhs = TRUE, multiple = TRUE, p.adj = "holm")$p
In 0.7.0, x1 remained below 0.05 (0.0465) even though x3 had a smaller raw 0.0819. In 0.7.1, x1 is 0.0819, matching stats::p.adjust().
What to do: re-run any Holm-adjusted Breusch-Pagan tests; the 0.7.1 values match stats::p.adjust().
Normality test: Cramér–von Mises
In 0.7.0, ols_test_normality() compared the Cramér–von Mises statistic with a standard uniform distribution instead of a normal distribution with mean and standard deviation estimated from the residuals, so it rejected normality for almost any regression. Version 0.7.1 calls nortest::cvm.test():
model <- lm(mpg ~ disp + hp + wt + qsec, data = mtcars) ols_test_normality(model)
-----------------------------------------------
Test Statistic pvalue
-----------------------------------------------
Shapiro-Wilk 0.9366 0.0600
Kolmogorov-Smirnov 0.1152 0.7464
Cramer-von Mises 0.0764 0.2230
Anderson-Darling 0.5859 0.1188
-----------------------------------------------
olsrr 0.7.0
| Cramér–von Mises W | 2.8122 |
| p-value | 0.0000 |
olsrr 0.7.1
| Cramér–von Mises W | 0.0764 |
| p-value | 0.2230 |
Same call on both CRAN versions: ols_test_normality(lm(mpg ~ disp + hp + wt + qsec, mtcars)). Changed values are highlighted.
# Run once with each version, e.g. in two sessions:
# remotes::install_version("olsrr", "0.7.0") # then
# install.packages("olsrr") # 0.7.1
library(olsrr)
model <- lm(mpg ~ disp + hp + wt + qsec, data = mtcars)
ols_test_normality(model)$cramer
In 0.7.1, the Cramér–von Mises result (p = 0.2230) agrees with nortest::cvm.test(residuals(model)) and aligns qualitatively with the Shapiro-Wilk (0.0600) and Anderson-Darling (0.1188) tests on the same residuals. (See What’s next about the Kolmogorov-Smirnov row.)
What to do: treat earlier Cramér–von Mises rejections from olsrr as unreliable and re-run the test.
Residual MSE and RMSE
In 0.7.0, ols_regress() and ols_model_performance() divided the residual sum of squares by lm(mpg ~ disp + hp + wt + qsec, data = mtcars) (MSE = 5.8011 and RMSE = 2.4085. Version 0.7.1 reports MSE = 6.8754 and RMSE = 2.6221, matching the ANOVA residual mean square and summary(model)$sigma.
olsrr 0.7.0
| MSE | 5.8011 |
| RMSE | 2.4085 |
olsrr 0.7.1
| MSE | 6.8754 |
| RMSE | 2.6221 |
Same call on both CRAN versions: ols_regress(lm(mpg ~ disp + hp + wt + qsec, mtcars)). Changed values are highlighted.
# Run once with each version, e.g. in two sessions:
# remotes::install_version("olsrr", "0.7.0") # then
# install.packages("olsrr") # 0.7.1
library(olsrr)
model <- lm(mpg ~ disp + hp + wt + qsec, data = mtcars)
r <- ols_regress(model)
c(mse = r$mse, rmse = r$rmse)
What to do: expect slightly larger MSE and RMSE values; they now match base R.
< section id="whats-new-since-2019" class="level2">What’s new since 2019
< section id="stepwise-selection-with-locked-predictors" class="level3">Stepwise Selection with Locked Predictors
Between versions 0.6.0 and 0.7.1, the variable selection family expanded to cover sbic), include and exclude arguments (by name or column index) so baseline covariates or confounders stay in the model throughout selection.
On the surgical dataset, locking age and alc_heavy via include keeps both predictors in the base model (Step 0, AIC = 795.956) and through every subsequent addition and removal step, even though age itself has
m_surg <- lm(
y ~ bcs + pindex + enzyme_test + liver_test + age + gender + alc_mod + alc_heavy,
data = surgical
)
ols_step_both_aic(m_surg, include = c("age", "alc_heavy"))
Stepwise Summary
------------------------------------------------------------------------------
Step Variable AIC SBC SBIC R2 Adj. R2
------------------------------------------------------------------------------
0 Base Model 795.956 803.912 637.696 0.17900 0.14680
1 liver_test (+) 763.110 773.055 606.421 0.56938 0.54354
2 enzyme_test (+) 752.416 764.350 596.755 0.65959 0.63180
3 pindex (+) 737.680 751.603 585.012 0.75030 0.72429
4 bcs (+) 732.494 748.406 581.938 0.78142 0.75351
5 liver_test (-) 732.339 746.262 580.934 0.77382 0.75026
------------------------------------------------------------------------------
Final Model Output
------------------
Model Summary
-------------------------------------------------------------------
R 0.880 RMSE 198.590
R-Squared 0.774 MSE 39438.163
Adj. R-Squared 0.750 Coef. Var 28.286
Pred R-Squared 0.688 AIC 732.339
MAE 140.528 SBC 746.262
-------------------------------------------------------------------
RMSE: Root Mean Square Error
MSE: Mean Square Error
MAE: Mean Absolute Error
AIC: Akaike Information Criteria
SBC: Schwarz Bayesian Criteria
ANOVA
-----------------------------------------------------------------------
Sum of
Squares DF Mean Square F Sig.
-----------------------------------------------------------------------
Regression 6476488.730 5 1295297.746 32.844 0.0000
Residual 1893031.807 48 39438.163
Total 8369520.537 53
-----------------------------------------------------------------------
Parameter Estimates
------------------------------------------------------------------------------------------------
model Beta Std. Error Std. Beta t Sig lower upper
------------------------------------------------------------------------------------------------
(Intercept) -1237.653 225.517 -5.488 0.000 -1691.085 -784.221
age -1.787 2.473 -0.050 -0.723 0.473 -6.759 3.184
alc_heavy 306.655 73.188 0.303 4.190 0.000 159.500 453.809
enzyme_test 11.244 1.314 0.601 8.555 0.000 8.601 13.887
pindex 10.062 1.633 0.428 6.163 0.000 6.779 13.344
bcs 81.587 17.870 0.329 4.566 0.000 45.657 117.516
------------------------------------------------------------------------------------------------
Ten-Panel Diagnostic Grid
ols_plot_diagnostics() gathers ten residual, fit, leverage, and influence views for an lm object: residuals vs fitted values, deleted studentized residuals vs predicted values, studentized residuals vs leverage, normal Q-Q plot, actual vs fitted values, Cook’s D chart, centered fit spread, residual spread, residual histogram, and residual box plot.
Calling ols_plot_diagnostics(model) directly pages the panels in 2×2 grids (and in 0.7.1 draws cleanly inside scripts, functions, and loops as well as at the top level). Setting print_plot = FALSE returns the ten ggplot objects without drawing them first, so they can be arranged in a single layout:
model <- lm(mpg ~ disp + hp + wt + qsec, data = mtcars) panels <- ols_plot_diagnostics(model, print_plot = FALSE) gridExtra::grid.arrange(grobs = panels, ncol = 2, top = "Regression Diagnostics")
Model Performance and Multi-Model Comparison
Added in 0.7.0 and refined in 0.7.1, ols_model_performance() reports fit metrics for a single model, while ols_compare_model_performance() places competing models side by side across sigma):
m1 <- lm(mpg ~ wt + hp, data = mtcars) m2 <- lm(mpg ~ wt + hp + qsec, data = mtcars) m3 <- lm(mpg ~ wt + hp + qsec + disp, data = mtcars) ols_compare_model_performance(m1, m2, m3)
name r2 r2_adj r2_pred aic sbc sbic rmse
1 m1 0.8267855 0.8148396 0.7810871 156.6523 162.5153 66.43956 2.593412
2 m2 0.8347678 0.8170643 0.7819955 157.1426 164.4713 67.43259 2.577785
3 m3 0.8351443 0.8107212 0.7710297 159.0696 167.8640 70.04081 2.622095
sigma
1 2.593412
2 2.577785
3 2.622095
Weighted Linear Regression in ols_regress()
In 0.7.1, ols_regress() supports weighted linear models fitted with lm(..., weights = ...), matching base R’s summary(), anova(), and confint() for coefficient estimates, standard errors,
d_wt <- mtcars d_wt$w <- seq_len(nrow(d_wt)) / 8 m_w1 <- lm(mpg ~ wt + hp, data = d_wt, weights = w) ols_regress(m_w1)
Model Summary
---------------------------------------------------------------
R 0.917 RMSE 3.876
R-Squared 0.841 MSE 15.027
Adj. R-Squared 0.830 Coef. Var 19.295
Pred R-Squared 0.786 AIC 167.360
MAE 1.953 SBC 173.223
---------------------------------------------------------------
RMSE: Root Mean Square Error
MSE: Mean Square Error
MAE: Mean Absolute Error
AIC: Akaike Information Criteria
SBC: Schwarz Bayesian Criteria
ANOVA
--------------------------------------------------------------------
Sum of
Squares DF Mean Square F Sig.
--------------------------------------------------------------------
Regression 2308.267 2 1154.134 76.806 0.0000
Residual 435.769 29 15.027
Total 2744.036 31
--------------------------------------------------------------------
Parameter Estimates
----------------------------------------------------------------------------------------
model Beta Std. Error Std. Beta t Sig lower upper
----------------------------------------------------------------------------------------
(Intercept) 37.938 1.545 24.560 0.000 34.779 41.098
wt -4.102 0.613 -0.666 -6.691 0.000 -5.355 -2.848
hp -0.030 0.008 -0.345 -3.678 0.001 -0.047 -0.013
----------------------------------------------------------------------------------------
Other improvements
ols_press()weights the squared deleted residuals for weighted models, sools_pred_rsq()no longer moves towards 1 when the weights are rescaled. For the weighted model shown above, multiplying the weights by 10 changed predicted from0.9077to0.9908in 0.7.0; in 0.7.1 both are0.7859.ols_plot_resid_hist()draws the bars with the same breaks as the normal curve. Previously the curve could be drawn at about half its correct height (0.54of the expected peak forlm(hwy ~ displ + cty + year, data = ggplot2::mpg)).ols_plot_comp_plus_resid()keeps every predictor for single-predictor and no-intercept models; 0.7.0 dropped a column and plotted a single point.ols_test_outlier()caps Bonferroni-adjusted -values at 1 (0.7.0 could report values such as6.62).- Information criteria count parameters with the model rank, which fixes factor predictors and rank-deficient models;
ols_aic(corrected = TRUE)matches the AICc formula (#224, #170, reported by @comatrion), and Mallows’ is correct for factor predictors (#92, reported by @cccfran) and inols_step_all_possible()(#215, reported by @LSantanaZA). - Stepwise selection handles transformed and factor predictors in refits (#244, #246), forward selection on saturated models (#199, reported by @hirscht), ties (#206, reported by @kaygmz) and aliased terms (#172, reported by @yayyb).
- Models fitted with
na.action = na.excludeor with weights are handled consistently across plots and tests; diagnostics whose formulas assume unweighted least squares now stop with a clear error for weighted fits (#236).
What’s next
Two issues remain open and will be fixed in 0.7.2. The Kolmogorov-Smirnov row of ols_test_normality() is not Lilliefors-corrected, so its nortest::lillie.test(residuals(model)). Two panels of ols_plot_diagnostics() also share the title “Residual Fit Spread Plot” (#253); the plotted values and axis labels are correct.
Acknowledgements
A big thank you to everyone who contributed to olsrr since 2019 by opening issues, reporting bugs and sending pull requests: @AminHP, @aravindhebbali, @ashimkapoor, @bappa10085, @ButlerMSDA, @CarlosJara72, @comatrion, @davidreilly007, @DrMondesire, @grantbrown, @hirscht, @hkincaid, @iankelk, @jmberros, @jorgevallejo, @jst04004, @jtheog, @kaygmz, @linusjf, @LSantanaZA, @modche, @Molina-Valero, @mwm-cbus, @njrockwood, @paleolimbot, @PsychlytxTD, @qwertytam, @sblanco74, @sckott, @SunsetBlvd, @tikuma-lsuhsc, @vasili111, @viqule11, @williamlai2, and @yayyb.
R-bloggers.com offers daily e-mail updates about R news and tutorials about learning R and many other topics. Click here if you're looking to post or find an R/data-science job.
Want to share your content on R-bloggers? click here if you have a blog, or here if you don't.
