Site icon R-bloggers

Juxtapose ML models in the Arena. Let the most credible one win!

[This article was first published on Stories by Przemyslaw Biecek on Medium, and kindly contributed to R-bloggers]. (You can report issue about the content on this page here)
Want to share your content on R-bloggers? click here if you have a blog, or here if you don't.
The Arena performs detailed comparative analysis of the ML models regardless of their internal structure or the language in which they are trained.

TL;DR: Piotr Piątyszek from MI2DataLab developed a new R package for interactive juxtapositioning of multiple ML models: http://arenar.drwhy.ai/

Most predictive ML models are based on a simple assumption: the future will be similar to the past. We can learn some relations on historical data and use them to predict the future.

The COVID19 pandemic shows us how fragile this assumption is.

Explainability is now more important than ever because without understanding how black box ML models work we risk meaningless predictions due to data drift, out of distribution errors or other issues.

As part of the DrWhy initiative, we are developing a new tool for interpretable interactive comparisons of multiple predictive models.

Code name: Arena

Arena is a new tool with interactive, vuejs powered frontend that allows to explore and compare any model regardless of its internal structure. The backend is implemented in the R package ArenaR.

Various XAI techniques are implemented, so one can juxtapose explanations for different models or explanations for different instances in an interactive dashboard.

Example exploration of glm, gbm and ranger models for titanic data.

Try it yourself

You can use Arena in three steps.

  1. Train models in any ML framework.
  2. Wrap them up with DALEX::explain function.
  3. Use ArenaR to automatically generate a dashboard to explore them.

You can install a dev version of ArenaR from the GitHub

devtools::install_github("ModelOriented/ArenaR")

Find the GitHub repository at https://github.com/ModelOriented/ArenaR.

Find vignettes and documentation at https://arenar.drwhy.ai/

Find a step by step introduction at https://arenar.drwhy.ai/articles/arena_intro_titanic.html

And feel free to star the repo for future updates.

To leave a comment for the author, please follow the link and comment on their blog: Stories by Przemyslaw Biecek on Medium.

R-bloggers.com offers daily e-mail updates about R news and tutorials about learning R and many other topics. Click here if you're looking to post or find an R/data-science job.
Want to share your content on R-bloggers? click here if you have a blog, or here if you don't.