# 2221 search results for "Regression"

## Kickin’ it with elastic net regression

With the kind of data that I usually work with, overfitting regression models can be a huge problem if I'm not careful. Ridge regression is a really effective technique for thwarting overfitting. It does this by penalizing the L2 norm… Continue reading →

## Evaluating Logistic Regression Models

August 17, 2015
By

Logistic regression is a technique that is well suited for examining the relationship between a categorical response variable and one or more categorical or continuous predictor variables. The model is generally presented in the following format, where β refers to the parameters and x represents the independent variables. log(odds)=β0+β1∗x1+...+βn∗xn The log(odds), or log-odds ratio, is defined

## R, Python, and SAS: Getting Started with Linear Regression

August 16, 2015
By

Consider the linear regression model, \$\$ y_i=f_i(boldsymbol{x}|boldsymbol{beta})+varepsilon_i, \$\$ where \$y_i\$ is the response or the dependent variable at the \$i\$th case, \$i=1,cdots, N\$ and the predictor or the independent variable is the \$boldsymbol{x}\$ term defined in the mean function \$f_i(boldsymbol{x}|boldsymbol{beta})\$. For simplicity, consider the following simple linear regression (SLR) model, \$\$ y_i=beta_0+beta_1x_i+varepsilon_i. \$\$ To obtain the (best) estimate...

## Bivariate Linear Regression

August 13, 2015
By

Regression is one of the – maybe even the single most important fundamental tool for statistical analysis in quite a large number of research areas. It forms the basis of many of the fancy statistical methods currently en vogue in the social sciences. Multilevel analysis and structural equation modeling are perhaps the most widespread and

## A glimpse on Gaussian process regression

August 11, 2015
By

The initial motivation for me to begin reading about Gaussian process (GP) regression came from Markus Gesmann’s blog entry about generalized linear models in R. The class of models implemented or available with the glm function in R comprises several interesting members that are standard tools in machine learning and data science, e.g. the logistic

## A glimpse on Gaussian process regression

August 11, 2015
By

The initial motivation for me to begin reading about Gaussian process (GP) regression came from Markus Gesmann’s blog entry about generalized linear models in R. The class of models implemented or available with the glm function in R comprises several interesting members that are standard tools in machine learning and data science, e.g. the logistic

## Simple regression models in R

August 1, 2015
By

Linear regression models are one the simplest and yet a very powerful models you can use in R to fit observed data and try to predict quantitative phenomena. Say you know that a certain variable y is somewhat correlated with a certain variable x and you can reasonably get an idea of what y would be given x....

## Empirical bias analysis of random effects predictions in linear and logistic mixed model regression

July 30, 2015
By

In the first technical post in this series, I conducted a numerical investigation of the biasedness of random effect predictions in generalized linear mixed models (GLMM), such as the ones used in the Surgeon Scorecard, I decided to undertake two explorations: firstly, the behavior of these estimates as more and more data are gathered for each

## Empirical bias analysis of random effects predictions in linear and logistic mixed model regression

July 30, 2015
By

In the first technical post in this series, I conducted a numerical investigation of the biasedness of random effect predictions in generalized linear mixed models (GLMM), such as the ones used in the Surgeon Scorecard, I decided to undertake two explorations: firstly, the behavior of these estimates as more and more data are gathered for each

## Regression with Multicollinearity Yields Multiple Sets of Equally Good Coefficients

July 6, 2015
By

The multiple regression equation represents the linear combination of the predictors with the smallest mean-squared error. That linear combination is a factorization of the predictors with the factors equal to the regression weights. You may see the wo...