Compute R2s and other performance indices for all your models!

April 29, 2019
By

(This article was first published on R on easystats, and kindly contributed to R-bloggers)

Indices of model performance (i.e., model quality, goodness of fit, predictive accuracy etc.) are very important, both for model comparison and model description purposes. However, their computation or extraction for a wide variety of models can be complex.

To address this, please let us introduce the performance package!

performance

We have recently decided to collaborate around the new easystats project, a set of packages designed to make your life easier (currently WIP). This project encompasses several packages, devoted for instance to model access or Bayesian analysis, as well as indices of model performance.

The goal of performance is to provide lightweight tools to assess and check the quality of your model. It includes functions such as R2 for many models (including logistic, mixed and Bayesian models), ICC or helpers to check convergence, overdipsersion or zero-inflation. See the list of functions here.

performance can be installed as follows:

install.packages("performance")  # Install the package
library(performance)  # Load it

Examples

Mixed Models

First, we calculate the r-squared value and intra-class correlation coefficient (ICC) for a mixed model, using r2() and icc(). r2() internally calls the appropriate function for the given model. In case of mixed models this will be r2_nakagawa(). r2_nakagawa() computes the marginal and conditional r-squared values, while icc() calculates an adjusted and conditional ICC, both based on the proposals from Nakagawa et al. 2017. For more details on the computation of the variances, see get_variance().

# Load the lme4 package
library(lme4)

# Fit a mixed model
model <- lmer(Sepal.Width ~ Petal.Length + (1|Species), data = iris)

# compute R2, based on Nakagawa et al. 2017
r2(model)
> # R2 for mixed models
> 
>   Conditional R2: 0.913
>      Marginal R2: 0.216
# compute intra-class correlation coefficient (ICC)
icc(model)
> # Intraclass Correlation Coefficient
> 
>      Adjusted ICC: 0.889
>   Conditional ICC: 0.697

Now let’s compute all available indices of performance appropriate for a given model. This can be done via the model_performance().

# Compute all performance indices
model_performance(model)
>   AIC BIC R2_conditional R2_marginal ICC_adjusted ICC_conditional RMSE
> 1 107 119           0.91        0.22         0.89             0.7 0.31

Bayesian Mixed Models

For Bayesian mixed models, we have the same features available (r-squared, ICC, …). In this example, we focus on the output from model_performance() only.

# Load the rstanarm package
library(rstanarm)

# Fit a Bayesian mixed model
model <- stan_glmer(Sepal.Width ~ Petal.Length + (1|Species), data = iris)

# Compute performance indices
model_performance(model)
>   ELPD ELPD_SE LOOIC LOOIC_SE R2_Median R2_MAD R2_Mean R2_SD R2_MAP
> 1  -44      10    87       20      0.47  0.045    0.47 0.046   0.47
>   R2_CI_low R2_CI_high R2_marginal_Median R2_marginal_MAD R2_marginal_Mean
> 1       0.4       0.55               0.27           0.045             0.27
>   R2_marginal_SD R2_marginal_MAP R2_marginal_CI_low R2_marginal_CI_high
> 1          0.048            0.29               0.19                0.34
>   R2_LOO_adjusted
> 1            0.45

More details about performance’s features are comming soon, stay tuned 😉

Get Involved

There is definitely room for improvement, and some new exciting features are already planned. Feel free to let us know how we could further improve this package!

To conclude, note that easystats is a new project in active development. Thus, do not hesitate to contact us if you want to get involved 🙂

  • Check out our other blog posts here!

To leave a comment for the author, please follow the link and comment on their blog: R on easystats.

R-bloggers.com offers daily e-mail updates about R news and tutorials on topics such as: Data science, Big Data, R jobs, visualization (ggplot2, Boxplots, maps, animation), programming (RStudio, Sweave, LaTeX, SQL, Eclipse, git, hadoop, Web Scraping) statistics (regression, PCA, time series, trading) and more...



If you got this far, why not subscribe for updates from the site? Choose your flavor: e-mail, twitter, RSS, or facebook...

Comments are closed.

Search R-bloggers

Sponsors

Never miss an update!
Subscribe to R-bloggers to receive
e-mails with the latest R posts.
(You will not see this message again.)

Click here to close (This popup will not appear again)