Done with convergence warnings? Publications that used allFit to compare optimisers

Development

A function name is a poor search term for bibliographic databases, because it appears in the methods section of an article and almost never in its title, abstract or keywords. Three sources with different coverage were therefore searched with a script in the website’s repository.

Scopus, which I queried through my scopusflow package (Bernabeu, 2026), indexes titles, abstracts, keywords, references and a few other fields but not the body of an article. A search for the function name in any indexed field can only find the articles that mention it in one of those fields. A second Scopus query serves as a proxy. It looks for titles, abstracts and keywords that mention lme4, lmerTest or brms alongside maximal random slopes and convergence. The assumption is that an abstract discussing convergence in that much detail probably belongs to a study that dealt with it. Europe PMC holds the full text of a large body of life-science articles, most but not all of them open access, and its search accepts full-text queries. OpenAlex offers a full-text search over the articles whose text it holds. Both full-text queries pair the function name with a mixed-model term, which removes some unrelated uses of the string without removing all of them.

library(scopusflow) # reads the key from SCOPUS_API_KEY
scopus_any_field <- scopus_fetch('ALL("allFit")')
scopus_proxy <- scopus_fetch(
paste('("lme4" OR "lmerTest" OR "brms") AND maximal AND "random slopes"',
'AND (convergence OR converge OR converged OR converging)'),
field = 'TITLE-ABS-KEY')

The two full-text queries are read from the files saved with the results, so they are exactly the queries that ran. A query retyped into a post can drift from the one that produced the counts, and this reads it back from disk instead.

cat(readLines('searches/europepmc_query.txt'), sep = '\n')
"allFit" AND (lme4 OR "mixed-effects" OR "mixed effects" OR "mixed model" OR "mixed models" OR "multilevel")
cat(readLines('searches/openalex_filter.txt'), sep = '\n')
fulltext.search:allFit AND (lme4 OR lmer OR "mixed effects" OR "mixed-effects" OR "mixed model" OR "mixed models" OR multilevel)

Results

Each search saved its results to a CSV file, and Tables 1 and 2 are built from those files. Nothing in this section queries a database, so the counts are those of the search date rather than of today.

library(dplyr)
library(ggplot2)
read_if_present <- function(file, source) {
path <- file.path('searches', file)
if (!file.exists(path)) return(NULL)
x <- read.csv(path, stringsAsFactors = FALSE)
x$source <- source
x
}
retrieved <- readLines('searches/retrieved.txt')
scopus_any <- read_if_present('scopus_allfit_any_field.csv', 'Scopus, any indexed field')
scopus_proxy <- read_if_present('scopus_convergence_proxy.csv', 'Scopus, convergence proxy')
europepmc <- read_if_present('europepmc_allfit_fulltext.csv', 'Europe PMC, full text')
openalex <- read_if_present('openalex_allfit_fulltext.csv', 'OpenAlex, full text')
standardise <- function(x) {
if (is.null(x)) return(NULL)
tibble(source = x$source,
doi = tolower(sub('^https?://doi.org/', '', x$doi)),
year = as.integer(x$year),
authors = x$authors,
title = x$title,
venue = if ('journal' %in% names(x)) x$journal else x$publication)
}
records <- bind_rows(lapply(list(scopus_any, scopus_proxy, europepmc, openalex),
standardise))
records |> count(source, name = 'records')
#> # A tibble: 4 × 2
#> source records
#> <chr> <int>
#> 1 Europe PMC, full text 30
#> 2 OpenAlex, full text 68
#> 3 Scopus, any indexed field 176
#> 4 Scopus, convergence proxy 1
# Count the Scopus any-field titles that mention a mixed model at all.
mixed_terms <- 'mixed|lme4|multilevel|random effect|convergence|optimi[sz]'
sum(grepl(mixed_terms, scopus_any$title, ignore.case = TRUE))
#> [1] 0
head(sort(table(scopus_any$publication), decreasing = TRUE), 4)
#>
#> European Journal of Pharmacology Journal of Medicinal Chemistry
#> 8 7
#> Neuropharmacology Psychopharmacology
#> 7 7

The three sources were queried on 2 September 2026. The Scopus search for the function name in any indexed field shows why a bare string makes a poor query. It returned 176 records, the largest group of them in pharmacology and medicinal chemistry, and not one of their titles mentions a mixed model. Scopus appears to match the string loosely against unrelated text, so those records are left out of the list below. The proxy query returned 1 record, which shows how rarely an abstract describes the treatment of convergence in that much detail. The two full-text sources come closest to answering the question, because they reach the methods section, where the function name usually appears. Their results are merged on the DOI and screened once, by dropping any record published before allFit existed, as a match in such an article cannot be a use of the function.

# allFit appeared in lme4's GitHub repository in April 2014 and first shipped
# with lme4 1.1-8 in June 2015, so a record published before 2014 cannot be a
# use of it, whatever matched the string.
allfit_year <- 2014L
used_allfit <- records |>
filter(source %in% c('Europe PMC, full text', 'OpenAlex, full text')) |>
mutate(key = ifelse(is.na(doi) | doi == '', paste(source, title), doi)) |>
group_by(key) |>
summarise(year = first(na.omit(year)), authors = first(authors),
title = first(title), venue = first(na.omit(venue)),
doi = first(doi), sources = paste(sort(unique(source)), collapse = '; '),
.groups = 'drop') |>
arrange(desc(year), authors)
too_early <- used_allfit |> filter(!is.na(year), year < allfit_year)
used_allfit <- used_allfit |> filter(is.na(year) | year >= allfit_year)
c(matched = nrow(used_allfit) + nrow(too_early),
before_allfit_existed = nrow(too_early), kept = nrow(used_allfit))
#> matched before_allfit_existed kept
#> 82 6 76
table(used_allfit$sources)
#>
#> Europe PMC, full text Europe PMC, full text; OpenAlex, full text
#> 13 16
#> OpenAlex, full text
#> 47

Figure 1 shows how many of the kept publications appeared in each year. Only the two full-text sources contribute to it, because the Scopus queries search indexed fields rather than the body of an article.

per_year <- used_allfit |>
filter(!is.na(year)) |>
count(year)
# The year of the search was still under way, so its label says where the
# count stops.
searched <- as.Date(substr(retrieved, 1, 10))
year_label <- function(y) {
ifelse(y == as.integer(format(searched, '%Y')),
paste0(y, '\nto ', trimws(format(searched, '%e %b'))), y)
}
ggplot(per_year, aes(year, n)) +
geom_col(fill = '#0072B2', width = 0.8) +
geom_text(aes(label = n), vjust = -0.5, size = 3.4) +
scale_x_continuous(breaks = min(per_year$year):max(per_year$year),
labels = year_label) +
scale_y_continuous(expand = expansion(mult = c(0, 0.1))) +
labs(x = NULL, y = NULL, title = 'Publications mentioning allFit') +
theme_minimal(base_size = 12) +
theme(plot.title.position = 'plot', plot.title = element_text(size = rel(1)),
axis.text.y = element_blank(), panel.grid = element_blank())

Figure 1: Publications per Year Whose Full Text Mentions allFit, According to Europe PMC and OpenAlex. The bar for the current year stops at the date of the search.

Table 1 lists the same publications, most recent first, with the sources in which each was found. Authors are given as the sources returned them. The two sources overlap in a minority of records, which says as much about their coverage as about the literature.

short_authors <- function(a) {
a <- ifelse(is.na(a), '', a)
first <- sub('[;,].*$', '', a)
ifelse(grepl('[;,]', a), paste(first, 'et al.'), first)
}
used_allfit |>
transmute(Year = year,
Authors = short_authors(authors),
Title = title,
Venue = venue,
DOI = ifelse(is.na(doi) | doi == '', '',
sprintf('[%s](https://doi.org/%s)', doi, doi)),
Sources = sources) |>
knitr::kable(caption = 'Candidate Publications Whose Full Text Mentions allFit, Most Recent First, With the Source in Which Each Was Found.')

The only hit of the proxy query, in Table 2, is an erratum notice that carries the abstract of the original article. A wider proxy query would be a reasonable place to look for further precedents, since studies that compared optimisers rarely name the function in a field that Scopus indexes.

records |>
filter(source == 'Scopus, convergence proxy') |>
transmute(Year = year, Authors = short_authors(authors), Title = title, Venue = venue,
DOI = ifelse(is.na(doi) | doi == '', '',
sprintf('[%s](https://doi.org/%s)', doi, doi))) |>
knitr::kable(caption = 'The Single Record Returned by the Scopus Convergence Proxy Query.')

Discussion

The count is a lower bound, because the searches miss part of the literature. Europe PMC holds full text mostly for open-access articles in journals within its scope, so a paywalled study in a linguistics or psychology journal is usually invisible to it unless a preprint or an accepted manuscript was deposited. OpenAlex holds the text of a subset of the literature that it does not fully document, and Scopus cannot see the methods section at all. None of the three can find a study that compared optimisers with its own loop or by calling the optimx package directly, or one that reported the comparison only in supplementary material. At the same time, a full-text match can be an article that uses the function for something other than a convergence check, or that mentions it only in a footnote. Records published before the function existed have been dropped, but the table has not otherwise been screened by hand, so some of the publications it lists may mention the function without having used it. The list is therefore a starting point for the reader who needs a precedent.

References

Barr, D. J., Levy, R., Scheepers, C., & Tily, H. J. (2013). Random effects structure for confirmatory hypothesis testing: Keep it maximal. Journal of Memory and Language, 68(3), 255–278. https://doi.org/10.1016/j.jml.2012.11.001

Bates, D., Kliegl, R., Vasishth, S., & Baayen, H. (2015). Parsimonious mixed models [Preprint]. arXiv. https://doi.org/10.48550/arXiv.1506.04967

Bates, D., Mächler, M., Bolker, B., & Walker, S. (2015). Fitting linear mixed-effects models using lme4. Journal of Statistical Software, 67(1), 1–48. https://doi.org/10.18637/jss.v067.i01

Bernabeu, P. (2026). scopusflow: A reproducible workflow layer for Scopus bibliographic searches (Version 0.4.0) [Computer software]. CRAN. https://doi.org/10.32614/CRAN.package.scopusflow

Brauer, M., & Curtin, J. J. (2018). Linear mixed-effects models and the analysis of nonindependent data: A unified framework to analyze categorical and continuous independent variables that vary within-subjects and/or within-items. Psychological Methods, 23(3), 389–411. https://doi.org/10.1037/met0000159

Matuschek, H., Kliegl, R., Vasishth, S., Baayen, H., & Bates, D. (2017). Balancing Type I error and power in linear mixed models. Journal of Memory and Language, 94, 305–315. https://doi.org/10.1016/j.jml.2017.01.001

Singmann, H., & Kellen, D. (2019). An introduction to mixed models for experimental psychology. In D. Spieler & E. Schumacher (Eds.), New methods in cognitive psychology (pp. 4–31). Routledge. https://doi.org/10.4324/9780429318405-2