Articles by rstats on Bryan Shalloway's Blog

Quantile Regression Forests for Prediction Intervals

April 20, 2021 | rstats on Bryan Shalloway's Blog

Quantile Regression Example Quantile Regression Forest Review Performance Coverage Interval Width Closing Notes Appendix Residual Plots Other Charts In this post I will build prediction intervals using quantile regression, more specifically, quantile regression forests. This is my third post on prediction intervals. Prior posts: Understanding Prediction Intervals (Part 1) Simulating Prediction ...
[Read more...]

Simulating Prediction Intervals

April 4, 2021 | rstats on Bryan Shalloway's Blog

Rough Idea Inspiration Procedure Example Simulate Prediction Interval Review Interval Width Coverage Closing Notes Appendix Conformal Inference Other Examples Using Simulation Confusion With Confidence Intervals Adjusting Procedure Alternative Procedure With CV Part 1 of my series of posts on building prediction intervals used data held-out from model training to evaluate the ...
[Read more...]

Understanding Prediction Intervals

March 17, 2021 | rstats on Bryan Shalloway's Blog

Providing More Than Point Estimates Considering Uncertainty Observation Specific Intervals A Few Things to Know About Prediction Intervals Prediction Intervals and Confidence Intervals Analytic Method of Calculating Prediction Intervals Visual Comparison of Prediction Intervals and Confidence Intervals Inference or Prediction? Cautions With Overfitting Generalizability Review Prediction Intervals Coverage Interval Width ...
[Read more...]

Weighting Confusion Matrices by Outcomes and Observations

December 7, 2020 | rstats on Bryan Shalloway's Blog

Model Performance Metrics Lending Data Example Starter Code Weighting by Classification Outcomes Metrics Across Decision Thresholds Weighting by Observations Closing note Appendix Weights of Observations During and Prior to Modeling Notes on Cost Sensitive Classification Weighted Classification Metrics Questions on Cost Sensitive Classification Arriving at Weights Weighting in predictive modeling ...
[Read more...]

Weighting Confusion Matrices by Outcomes and Observations

December 7, 2020 | rstats on Bryan Shalloway's Blog

Model Performance Metrics Lending Data Example Starter Code Weighting by Classification Outcomes Metrics Across Decision Thresholds Weighting by Observations Closing note Appendix Weights of Observations During and Prior to Modeling Notes on Cost Sensitive Classification Weighted Classification Metrics Questions on Cost Sensitive Classification Arriving at Weights Weighting in predictive modeling ...
[Read more...]

Undersampling Will Change the Base Rates of Your Model’s Predictions

November 22, 2020 | rstats on Bryan Shalloway's Blog

Create Data Association of ‘feature’ and ‘target’ Resample Build Models Rescale Predictions to Predicted Probabilities Appendix Density Plots Lift Plot Comparing Scaling Methods TLDR: In classification problems, under and over sampling1 techniques shift the distribution of predicted probabilities towards the minority class. If your problem requires accurate probabilities you will ...
[Read more...]

Undersampling Will Change the Base Rates of Your Model’s Predictions

November 22, 2020 | rstats on Bryan Shalloway's Blog

Create Data Association of ‘feature’ and ‘target’ Resample Build Models Rescale Predictions to Predicted Probabilities Appendix Density Plots Lift Plot TLDR: In classification problems, under and over sampling1 techniques shift the distribution of predicted probabilities towards the minority class. If your problem requires accurate probabilities you will need to adjust ...
[Read more...]
1 2

Never miss an update!
Subscribe to R-bloggers to receive
e-mails with the latest R posts.
(You will not see this message again.)

Click here to close (This popup will not appear again)