Subsetting in the presence of NAs

October 6, 2018

(This article was first published on R – Statistical Odds & Ends, and kindly contributed to R-bloggers)

In R, we can subset a data frame df easily by putting the conditional in square brackets after df. For example, if I want all the rows in df which have value equal to 1 in the column colA, all I have to do is

df[df$colA == 1, ]

Recently, I realized that this approach can be problematic when there are NAs present in the data! For example, let df be the following data frame:

df <- data.frame(colA = c(1, 2, 3, NA, NA),
                 colB = c("a", "b", "c", "d", NA))

# co1A colB
# 1    1    a
# 2    2    b
# 3    3    c
# 4   NA    d
# 5   NA 

If I want to pull out all rows such that the value in colA is equal to 2, I would use the following code expecting just one row to be returned:

df[df$colA == 2, ]

#      colA colB
# 2       2    b
# NA     NA 
# NA.1   NA 

All the rows with NA in colA also got included, and all the values in those rows got converted into NAs! To avoid these rows from being added, we have to explicitly check that the values are not NAs:

df[!$colA) & df$colA == 2, ]

# colA colB
# 2    2    b

dplyr acts a bit differently in that it removes NAs by default and so you do not have to check for them:

df %>% filter(colA == 2)

# colA colB
# 2    2    b

To leave a comment for the author, please follow the link and comment on their blog: R – Statistical Odds & Ends. offers daily e-mail updates about R news and tutorials on topics such as: Data science, Big Data, R jobs, visualization (ggplot2, Boxplots, maps, animation), programming (RStudio, Sweave, LaTeX, SQL, Eclipse, git, hadoop, Web Scraping) statistics (regression, PCA, time series, trading) and more...

If you got this far, why not subscribe for updates from the site? Choose your flavor: e-mail, twitter, RSS, or facebook...

Comments are closed.

Search R-bloggers


Never miss an update!
Subscribe to R-bloggers to receive
e-mails with the latest R posts.
(You will not see this message again.)

Click here to close (This popup will not appear again)