Articles by jmount

abs and relu are not Mercer Kernels

December 25, 2020 | jmount

I am sharing some rough notes (in R and Python) here on how while dot(a, b) fulfills “Mercer’s condition” (by definition!, and I’ll just informally call these beasts a “Mercer Kernel”), the seemingly harmless variations abs(dot(a, b)) relu(dot(a, b)) are not Mercer Kernels (... [Read more...]

Bounding Excess Generalization Error

December 23, 2020 | jmount

I am sharing a new free video where I work through a great common argument that bounds expected excess generalization error as a ratio of model complexity (in rows) over training set size (again in rows), independent of problem dimension. (link) For more of my notes on support vector machines […] [Read more...]

My Opinion on R’s Upcoming Pipe

December 7, 2020 | jmount

R‘s upcoming pipe appears to be currently proposed as a syntactic transform of the form: a |__ f(...) -__ f(a, ...) a |__ f() -__ f(a) There is a current active discussion on this prototype and some interesting points come up. Note the current proposal appears to disallow a |__ […] [Read more...]

R is Getting an Official Pipe Operator

December 5, 2020 | jmount

It looks like R is getting an official pipe operator (ref). R doesn’t work under an RFC process, so we hear about these things and they are discussed on the R-devel mailing list. I’ve written on this topic before (ref), and I have taped some new comments. (link) ... [Read more...]

BARUG ROC day invitation

November 4, 2020 | jmount

I’ve recorded a video invitation to help encourage you to consider attending BARUG’s online ROC day (Tuesday, November 10, 2020 4:30 PM US Pacific time). Please check it out and share. (link) [Read more...]

A Single Parameter Family Characterizing Probability Model Performance

October 29, 2020 | jmount

Introduction We’ve been writing on the distribution density shapes expected for probability models in ROC (receiver operator characteristic) plots, double density plots, and normal/logit-normal densities frameworks. I thought I would re-approach the issue with a specific family of examples. Definitions Let’s define a “probability model” as a ...
[Read more...]

The Double Density Plot Contains a Lot of Useful Information

October 27, 2020 | jmount

The double density plot contains a lot of useful information. This is a plot that shows the distribution of a continuous model score, conditioned on the binary categorical outcome to be predicted. As with most density plots: the y-axis is an abstract quantity called density picked such that the area […]
[Read more...]

Your Lopsided Model is Out to Get You

October 26, 2020 | jmount

For classification problems I argue one of the biggest steps you can take to improve the quality and utility of your models is to prefer models that return scores or return probabilities instead of classification rules. Doing this also opens a second large opportunity for improvement: working with your domain […] [Read more...]

The Shift and Balance Fallacies

October 15, 2020 | jmount

Two related fallacies I see in machine learning practice are the shift and balance fallacies (for an earlier simple fallacy, please see here). They involve thinking logistic regression has a bit simpler structure that it actually does, and also thinking logistic regression is a bit less powerful than it actually […] [Read more...]

Surgery on ROC Plots

October 13, 2020 | jmount

This note is a little break from our model homotopy series. I have a neat example where one combines two classifiers to get a better classifier using a method I am calling “ROC surgery.” In ROC surgery we look at multiple ROC plots and decide we want to cut out […]
[Read more...]

Model Homotopies in the Wild

October 12, 2020 | jmount

So are model homotopies commonly used? Yes, they are. As an example consider glmnet: Jerome Friedman, Trevor Hastie, Robert Tibshirani (2010). Regularization Paths for Generalized Linear Models via Coordinate Descent. Journal of Statistical Software, 33(1), 1-22. URL From help(glmnet): library(glmnet) x = matrix(rnorm(100 * 20), 100, 20) g2 = […]
[Read more...]

Tailored Models are Not The Same as Simple Corrections

October 11, 2020 | jmount

Let’s take a stab at our first note on a topic that pre-establishing the definitions of probability model homotopy makes much easier to write. In this note we will discuss tailored probability models. There are models deliberately fit to training data that has an outcome prevalence equal to the ... [Read more...]

How to Pick an Optimal Utility Threshold Using the ROC Plot

October 10, 2020 | jmount

Nina Zumel just completed an excellent short sequence of articles on picking optimal utility thresholds to convert a continuous model score for a classification problem into a deployable classification rule. Squeezing the Most Utility from Your Models Estimating Uncertainty of Utility Curves This is very compatible with our advice to […]
[Read more...]
1 2 3

Never miss an update!
Subscribe to R-bloggers to receive
e-mails with the latest R posts.
(You will not see this message again.)

Click here to close (This popup will not appear again)