Example 2014.5: Simple mean imputation

April 25, 2014
By

(This article was first published on SAS and R, and kindly contributed to R-bloggers)

We're both users of multiple imputation for missing data. We believe it is the most practical principled method for incorporating the most information into data analysis. In fact, one of our more successful collaborations is a review of software for multiple imputation.

But, for me at least, there are times when a simpler form of imputation may be useful. For example, it may be desirable to calculate the mean of the observed values and substitute it for any missing values. Typically it would be unwise to attempt to use a data set completed in this way for formal inference, but it could be convenient under deadline pressure or for a very informal overview of the data.

Nick disagrees. He finds it hard to imagine any setting in which he would ever use such a primitive approach. He passes on to the reader the sage advice he received in graduate school: that making up data in such an ad-hoc fashion might be construed as dereliction or even misconduct.   Use of single imputation approaches (which yield bias in many settings and attenuate estimates of variance) seems hard to justify in 2014.  But one of the hallmarks of our partnership is that we can agree to disagree on an absolute ban, while jointly advising the reader to proceed with great caution.

SAS
In SAS, it would possible to approach this using proc means to find the means and then add them back into the data set in a data step. But there is a simpler way, using proc standard.
proc standard data=indata out=outdata replace; run;
This will replace the values of all missing numeric variables in the indata data set with the mean of the observed values, and save the result in a new data set, outdata. To restrict the operation to specific variables, use a var statement.

R
There may be a function designed to do this in R, but it's simple enough using the features of the language. We provide an option using the bracket ([) extractor operator and another using the ifelse() function. The latter may be more approachable for those less familiar with R.
df = data.frame(x = 1:20, y = c(1:10,rep(NA,10)))df$y[is.na(df$y)] = mean(df\$y, na.rm=TRUE)
# alternative
df = transform(df, y = ifelse(is.na(y), mean(y, na.rm=TRUE), y))

In the first example, we identify elements of y that are NA, and replace them with the mean, if so. In the second, we test each element of y; if it is NA, we replace with the mean, otherwise we replace with the original value.

R-bloggers.com offers daily e-mail updates about R news and tutorials on topics such as: visualization (ggplot2, Boxplots, maps, animation), programming (RStudio, Sweave, LaTeX, SQL, Eclipse, git, hadoop, Web Scraping) statistics (regression, PCA, time series, trading) and more...