# An example of base::split() for looping through groups

**Very statisticious on Very statisticious**, and kindly contributed to R-bloggers]. (You can report issue about the content on this page here)

Want to share your content on R-bloggers? click here if you have a blog, or here if you don't.

I recently had a question from a client about the simplest way to subset a data.frame and apply a function to each subset. “Simplest” could mean many things, of course, since what is simple for one person could appear very difficult to another. In this specific case I suggested using `base::split()`

as a possible option since it is one I find fairly approachable.

I turns out I don’t have a go-to example for how to get started with a `split()`

approach. So here’s a quick blog post about it! 😄

## Table of Contents

# Load R packages

I’ll load **purrr** for looping through lists.

`library(purrr) # 0.3.3`

# A dataset with groups

I made a small dataset to use with `split()`

. The `id`

variable contains the group information. There are three groups, a, b, and c, with 10 observations per group. There are also two numeric variables, `var1`

and `var2`

.

```
dat = structure(list(id = structure(c(1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L,
1L, 1L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 3L, 3L, 3L, 3L,
3L, 3L, 3L, 3L, 3L, 3L), .Label = c("a", "b", "c"), class = "factor"),
var1 = c(4, 2.7, 3.4, 2.7, 4.6, 2.9, 2.2, 4.5, 4.6, 2.4,
3, 3.8, 2.5, 4, 3.6, 2.7, 4.5, 4.1, 4.2, 2.2, 4.9, 4.4, 3.6,
3.3, 2.7, 3.9, 4.9, 4.9, 4.3, 3.4), var2 = c(6, 22.3, 19.4,
22.8, 18.6, 14.2, 10.9, 22.7, 22.4, 11.7, 6, 13.3, 12.5,
6.3, 13.6, 20.5, 23.6, 10.9, 8.9, 20.9, 23.7, 15.9, 22.1,
11.6, 22, 17.7, 21, 20.8, 16.7, 21.4)), class = "data.frame", row.names = c(NA,
-30L))
head(dat)
```

```
# id var1 var2
# 1 a 4.0 6.0
# 2 a 2.7 22.3
# 3 a 3.4 19.4
# 4 a 2.7 22.8
# 5 a 4.6 18.6
# 6 a 2.9 14.2
```

# Create separate data.frames per group

If the goal is to apply a function to each dataset in each group, we need to pull out a dataset for each `id`

. One approach to do this is to make a subset for each group and then apply the function of interest to the subset. A classic approach would be to do the subsetting within a `for()`

loop.

This is a situation where I find `split()`

to be really convenient. It splits the data by a defined group variable so we don’t have to subset things manually.

The output from `split()`

is a list. If I split a dataset by groups, each element of the list will be a data.frame for one of the groups. Note the group values are used as the names of the list elements. I find the list-naming aspect of `split()`

handy for keeping track of groups in subsequent steps.

Here’s an example, where I split `dat`

by the `id`

variable.

```
dat_list = split(dat, dat$id)
dat_list
```

```
# $a
# id var1 var2
# 1 a 4.0 6.0
# 2 a 2.7 22.3
# 3 a 3.4 19.4
# 4 a 2.7 22.8
# 5 a 4.6 18.6
# 6 a 2.9 14.2
# 7 a 2.2 10.9
# 8 a 4.5 22.7
# 9 a 4.6 22.4
# 10 a 2.4 11.7
#
# $b
# id var1 var2
# 11 b 3.0 6.0
# 12 b 3.8 13.3
# 13 b 2.5 12.5
# 14 b 4.0 6.3
# 15 b 3.6 13.6
# 16 b 2.7 20.5
# 17 b 4.5 23.6
# 18 b 4.1 10.9
# 19 b 4.2 8.9
# 20 b 2.2 20.9
#
# $c
# id var1 var2
# 21 c 4.9 23.7
# 22 c 4.4 15.9
# 23 c 3.6 22.1
# 24 c 3.3 11.6
# 25 c 2.7 22.0
# 26 c 3.9 17.7
# 27 c 4.9 21.0
# 28 c 4.9 20.8
# 29 c 4.3 16.7
# 30 c 3.4 21.4
```

# Looping through the list

Once the data are split into separate data.frames per group, we can loop through the list and apply a function to each one using whatever looping approach we prefer.

For example, if I want to fit a linear model of `var1`

vs `var2`

for each group I might do the looping with `purrr::map()`

or `lapply()`

.

Each element of the new list still has the grouping information attached via the list names.

`map(dat_list, ~lm(var1 ~ var2, data = .x) )`

```
# $a
#
# Call:
# lm(formula = var1 ~ var2, data = .x)
#
# Coefficients:
# (Intercept) var2
# 2.64826 0.04396
#
#
# $b
#
# Call:
# lm(formula = var1 ~ var2, data = .x)
#
# Coefficients:
# (Intercept) var2
# 3.80822 -0.02551
#
#
# $c
#
# Call:
# lm(formula = var1 ~ var2, data = .x)
#
# Coefficients:
# (Intercept) var2
# 3.35241 0.03513
```

I could also create a function that fit a model and then returned model output. For example, maybe what I really wanted to do is the fit a linear model and extract \(R^2\) for each group model fit.

```
r2 = function(data) {
fit = lm(var1 ~ var2, data = data)
broom::glance(fit)
}
```

The output of my `r2`

function, which uses `broom::glance()`

, is a data.frame.

`r2(data = dat)`

```
# # A tibble: 1 x 11
# r.squared adj.r.squared sigma statistic p.value df logLik AIC BIC
#
```
# 1 0.0292 -0.00550 0.867 0.841 0.367 2 -37.3 80.5 84.7
# # ... with 2 more variables: deviance , df.residual

Since the function output is a data.frame, I can use `purrr::map_dfr()`

to combine the output per group into a single data.frame. The `.id`

argument creates a new variable to store the list names in the output.

`map_dfr(dat_list, r2, .id = "id")`

```
# # A tibble: 3 x 12
# id r.squared adj.r.squared sigma statistic p.value df logLik AIC
#
```
# 1 a 0.0775 -0.0378 0.968 0.672 0.436 2 -12.7 31.5
# 2 b 0.0387 -0.0815 0.832 0.322 0.586 2 -11.2 28.5
# 3 c 0.0285 -0.0930 0.808 0.235 0.641 2 -10.9 27.9
# # ... with 3 more variables: BIC , deviance , df.residual

# Splitting by multiple groups

It is possible to split data by multiple grouping variables in the `split()`

function. The grouping variables must be passed as a list.

Here’s an example, using the built-in `mtcars`

dataset. I show only the first two list elements to demonstrate that the list names are now based on a combination of the values for the two groups. By default these values are separated by a `.`

(but see the `sep`

argument to control this).

```
mtcars_cylam = split(mtcars, list(mtcars$cyl, mtcars$am) )
mtcars_cylam[1:2]
```

```
# $`4.0`
# mpg cyl disp hp drat wt qsec vs am gear carb
# Merc 240D 24.4 4 146.7 62 3.69 3.190 20.00 1 0 4 2
# Merc 230 22.8 4 140.8 95 3.92 3.150 22.90 1 0 4 2
# Toyota Corona 21.5 4 120.1 97 3.70 2.465 20.01 1 0 3 1
#
# $`6.0`
# mpg cyl disp hp drat wt qsec vs am gear carb
# Hornet 4 Drive 21.4 6 258.0 110 3.08 3.215 19.44 1 0 3 1
# Valiant 18.1 6 225.0 105 2.76 3.460 20.22 1 0 3 1
# Merc 280 19.2 6 167.6 123 3.92 3.440 18.30 1 0 4 4
# Merc 280C 17.8 6 167.6 123 3.92 3.440 18.90 1 0 4 4
```

If all combinations of groups are not present, the `drop`

argument in `split()`

allows us to drop missing combinations. By default combinations that aren’t present are kept as 0-length data.frames.

# Other thoughts on split()

I feel like `split()`

was a gateway function for me to get started working with lists and associated convenience functions like `lapply()`

and `purrr::map()`

for looping through lists. I think learning to work with lists and “list loops” also made the learning curve for list-columns in data.frames and the `nest()`

/`unnest()`

approach of analysis-by-groups a little less steep for me.

# Just the code, please

Here’s the code without all the discussion. Copy and paste the code below or you can download an R script of uncommented code from here.

```
library(purrr) # 0.3.3
dat = structure(list(id = structure(c(1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L,
1L, 1L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 3L, 3L, 3L, 3L,
3L, 3L, 3L, 3L, 3L, 3L), .Label = c("a", "b", "c"), class = "factor"),
var1 = c(4, 2.7, 3.4, 2.7, 4.6, 2.9, 2.2, 4.5, 4.6, 2.4,
3, 3.8, 2.5, 4, 3.6, 2.7, 4.5, 4.1, 4.2, 2.2, 4.9, 4.4, 3.6,
3.3, 2.7, 3.9, 4.9, 4.9, 4.3, 3.4), var2 = c(6, 22.3, 19.4,
22.8, 18.6, 14.2, 10.9, 22.7, 22.4, 11.7, 6, 13.3, 12.5,
6.3, 13.6, 20.5, 23.6, 10.9, 8.9, 20.9, 23.7, 15.9, 22.1,
11.6, 22, 17.7, 21, 20.8, 16.7, 21.4)), class = "data.frame", row.names = c(NA,
-30L))
head(dat)
dat_list = split(dat, dat$id)
dat_list
map(dat_list, ~lm(var1 ~ var2, data = .x) )
r2 = function(data) {
fit = lm(var1 ~ var2, data = data)
broom::glance(fit)
}
r2(data = dat)
map_dfr(dat_list, r2, .id = "id")
mtcars_cylam = split(mtcars, list(mtcars$cyl, mtcars$am) )
mtcars_cylam[1:2]
```

**leave a comment**for the author, please follow the link and comment on their blog:

**Very statisticious on Very statisticious**.

R-bloggers.com offers

**daily e-mail updates**about R news and tutorials about learning R and many other topics. Click here if you're looking to post or find an R/data-science job.

Want to share your content on R-bloggers? click here if you have a blog, or here if you don't.