lexsync: From a word-frequency corpus to an EEG-ready experiment

Put the Design in Data

The design below asks for 40 words in each frequency condition, matched on length, neighbourhood density and OLD20. Those three controls and frequency itself are then balanced across two presentation lists. Frequency is expressed on the Zipf scale (van Heuven et al., 2014), and OLD20 measures orthographic similarity as the mean edit distance between a word and its 20 nearest neighbours (Yarkoni et al., 2008).

design <- list(
name = 'frequency_eeg',
language = 'english',
n_per_condition = 40,
pool_filters = list(length = c(3, 7), frequency = c(3.8, 7.0)),
conditions = list(
list(name = 'high_frequency', define_by = list(frequency = c(5.2, 7.0))),
list(name = 'low_frequency', define_by = list(frequency = c(3.8, 4.4)))
),
match_on = c('length', 'n_density', 'old20'),
counterbalance = list(
lists = 2,
optimise = TRUE,
balance_on = c('length', 'n_density', 'old20', 'frequency')
)
)
pool <- build_pool(lexicon, design$pool_filters)
candidate_counts <- vapply(
design$conditions,
function(condition) nrow(build_pool(pool, condition$define_by)),
integer(1)
)
names(candidate_counts) <- vapply(design$conditions, `[[`, character(1), 'name')
candidate_counts
#> high_frequency low_frequency
#> 71 480

The high-frequency band is the smaller candidate set, so it limits both the number and the quality of possible matches. Counting the candidates before matching shows whether a later shortfall stems from the size of this pool.

Select and Check the Words

match_stimuli() spreads anchor words across the smaller frequency band and finds nearby words from the other condition in the standardised space of the control variables. Standardising first means that a given distance counts the same whichever control it falls on.

stimuli <- match_stimuli(pool, design, schema)
head(stimuli[, c('set', 'condition', 'word', 'frequency', 'length',
'n_density', 'old20')])
#> set condition word frequency length n_density old20
#> 1 1 high_frequency knew 5.20 4 3 1.75
#> 2 2 high_frequency date 5.22 4 17 1.00
#> 3 3 high_frequency energy 5.23 6 0 2.60
#> 4 4 high_frequency points 5.23 6 5 1.65
#> 5 5 high_frequency running 5.23 7 3 1.80
#> 6 6 high_frequency term 5.23 4 4 1.65

The report treats matching as a claim of equivalence. For each control, it gives Cohen’s \(d\) with a 90% confidence interval and applies the two one-sided tests (TOST) procedure against the schema’s bound of half a standard deviation (Lakens, 2017). The frequency row reflects the intended manipulation, so Figure 1 leaves it out.

report <- match_report(
stimuli,
c('frequency', 'length', 'n_density', 'old20'),
schema
)
report$comparisons
#> condition reference dimension cohens_d d_ci_low d_ci_high var_ratio tost_p
#> 1 low_frequency high_frequency frequency 5.642 5.270 6.014 0.293 1.0000
#> 2 low_frequency high_frequency length 0.000 -0.372 0.372 1.000 0.0141
#> 3 low_frequency high_frequency n_density 0.076 -0.296 0.448 0.632 0.0308
#> 4 low_frequency high_frequency old20 0.016 -0.356 0.389 0.843 0.0168
#> equivalent
#> 1 FALSE
#> 2 TRUE
#> 3 TRUE
#> 4 TRUE
controls <- subset(report$comparisons, dimension != 'frequency')
controls$dimension <- factor(
controls$dimension,
levels = rev(c('length', 'n_density', 'old20')),
labels = rev(c('Word length', 'Neighbourhood density', 'OLD20'))
)
ggplot(controls, aes(cohens_d, dimension)) +
annotate('rect', xmin = -0.5, xmax = 0.5, ymin = -Inf, ymax = Inf,
fill = '#E8F1F5') +
geom_vline(xintercept = 0, colour = '#66747B', linewidth = 0.5) +
geom_pointrange(aes(xmin = d_ci_low, xmax = d_ci_high),
colour = '#006D77', linewidth = 0.8, size = 0.7) +
scale_x_continuous(
breaks = c(-0.5, 0, 0.5), expand = expansion(0),
labels = scales::label_number(accuracy = 0.1, style_negative = 'minus')
) +
coord_cartesian(xlim = c(-0.56, 0.56)) +
labs(x = 'Standardised difference (low minus high), 90% CI', y = NULL,
title = 'Realised balance on the control variables',
subtitle = 'Shading marks the prespecified equivalence region') +
theme_minimal(base_size = 12) +
theme(panel.grid = element_blank(),
plot.title.position = 'plot',
plot.title = element_text(size = 13, face = 'bold'),
plot.subtitle = element_text(size = 11, margin = margin(b = 10)),
axis.title.x = element_text(size = 10.5, margin = margin(t = 6)),
axis.text.x = element_text(size = 10),
axis.text.y = element_text(size = 11, hjust = 0, colour = 'grey20',
margin = margin(r = 10)),
plot.margin = margin(8, 10, 8, 8))

Figure 1: Realised Balance Between the Frequency Conditions on Each Control Variable, With 90% Confidence Intervals Against the Prespecified Equivalence Region.

The intervals deserve more attention than the point estimates, because they carry the uncertainty. With few items per condition, an estimate near zero can still be too imprecise to support a claim of equivalence.

Carry the Items Into the Experiment

The design also describes each trial, which presents a fixation cross, the word with a condition trigger locked to its onset, a response window and a blank interval of jittered length. The jitter is derived from a keyed hash of the design and seed, so rebuilding the experiment reproduces the same timing.

design$events <- list(
list(type = 'fixation', content = '+', duration_ms = 500),
list(type = 'text', content = '{word}', duration_ms = 800,
trigger = 'condition', onset_locked = TRUE),
list(type = 'response', keys = c('f', 'j'), timeout_ms = 2000),
list(type = 'blank', duration = list(jitter = c(400, 800), as = 'iti_ms'))
)
list_plan <- balance_lists(stimuli, design, schema)
trials <- counterbalance(stimuli, design, schema, list_plan$list_of_set)
trials <- resolve_trial_timing(trials, design, schema)
trials <- assign_triggers(trials)
c(cost_before = list_plan$report$cost_before,
cost_after = list_plan$report$cost_after,
swaps = list_plan$report$n_swaps)
#> cost_before cost_after swaps
#> 743080 17880 5
head(trials[, c('list', 'trial', 'condition', 'word', 'iti_ms',
'condition_trigger')])
#> list trial condition word iti_ms condition_trigger
#> 1 1 1 low_frequency approve 696 101
#> 2 1 2 high_frequency knew 504 102
#> 3 1 3 high_frequency mother 643 102
#> 4 1 4 low_frequency loop 688 101
#> 5 1 5 high_frequency lost 489 102
#> 6 1 6 low_frequency ghana 498 101

The imbalance cost sums each list’s departure from its fair share on every balanced variable, scaled to that variable’s mean. Each swap exchanged a matched word pair in one list for a pair in the other.

All three exported files read from this trial table. In the PsychoPy script, win.callOnFlip sends the trigger on the refresh that presents the word.

experiments <- export_experiments(trials, design, schema, output_dir)
basename(unlist(experiments))
#> [1] "frequency_eeg_english_psychopy.py" "frequency_eeg_english.osexp"
#> [3] "frequency_eeg_english.html"
psychopy_code <- readLines(experiments$psychopy, warn = FALSE)
grep('^\\s*(TRIGGER_HOLD_MS\\s*=|win\\.callOnFlip\\()', psychopy_code,
value = TRUE, perl = TRUE)
#> [1] "TRIGGER_HOLD_MS = 50"
#> [2] " win.callOnFlip(port.setData, trigger)"
#> [3] " win.callOnFlip(port.setData, 0)"
#> [4] " win.callOnFlip(port.setData, trig)"
#> [5] " win.callOnFlip(port.setData, BLOCK_START_TRIGGER)"
#> [6] " win.callOnFlip(port.setData, BLOCK_END_TRIGGER)"

Limits

Matching can only draw on what the candidate corpus contains. The bundled English lexicon takes its words and Zipf frequencies from wordfreq (Speer, 2022). It serves this demonstration, but it includes proper nouns, estimates syllables from spelling and has no part-of-speech information. A study should therefore supply its own exclusions and whatever norms it needs. Equivalence also depends on the bound chosen before selection, and the default of half a standard deviation is only a starting point. Each study should derive its own bound from the smallest difference that would matter for its research question (Lakens, 2017).

The R documentation and Python documentation cover pseudoword generation, semantic norms, continuous designs, Latin-square counterbalancing and the full file-based pipeline. Across the package, the design records the experiment as intended, while the generated tables and reports record it as realised.