How to Pick an Optimal Utility Threshold Using the ROC Plot

[This article was first published on R – Win Vector LLC, and kindly contributed to R-bloggers]. (You can report issue about the content on this page here)
Want to share your content on R-bloggers? click here if you have a blog, or here if you don't.

Nina Zumel just completed an excellent short sequence of articles on picking optimal utility thresholds to convert a continuous model score for a classification problem into a deployable classification rule.

This is very compatible with our advice to prefer continuous scoring models, and probability models in particular, to “hard” classification rules during model development for classification problems. In fact her point is the benefit: by delaying the converstion of a contiuous score to a classification rule you may be able to use better more business oriented criteria to make the conversion. And you may be able to change the rule when utilities ore prevelances change.

For completeness we are sharing an example of how you reproduce her calculation on the ROC plot itself. This is a bit of a “more standard way of doing things”, but I think in fact didactically inferior to her method of working more directly in terms of utilities. So to see how to work with the ROC plot, please check out our new note here.

Unnamed chunk 9 1

Figure from: “ROC optimization”

To leave a comment for the author, please follow the link and comment on their blog: R – Win Vector LLC. offers daily e-mail updates about R news and tutorials about learning R and many other topics. Click here if you're looking to post or find an R/data-science job.
Want to share your content on R-bloggers? click here if you have a blog, or here if you don't.

Never miss an update!
Subscribe to R-bloggers to receive
e-mails with the latest R posts.
(You will not see this message again.)

Click here to close (This popup will not appear again)