TLDR: This post estimates potential labor market transition pathways based on the similarity of skills, abilities and knowledge required by occupations. The approach broadly follows Bratanova et al. (2026). Like their research, the considered transition pathways are for workers currently employed as truck drivers. Unlike their approach, we’ve used a more recent version of the O*NET database and a different SOC-to-ANZSCO crosswalk. Although the point wasn’t to precisely replicate their results, our approach largely did, with differences in similarity scores likely stemming from our different mapping of the SOC to the ANZSCO rather than from the method itself.

Citation: A. Bratanova, C. Mason, D. Evans, E. Schleiger, E. Grimberg, G. Walker, H. Pham, K. Bulled, Truck drivers and automation: A methodology for identifying and supporting workforce transition in the Australian road freight sector, Transport Policy, Volume 176, 2026, 103907, ISSN 0967-070X,.

Note: Thanks to the author(s) for willingly sharing their data, code and responding to questions about the methodology. Any errors are my own.

Source Data: Correspondence files were developed based on the process outlined in this post, but can be downloaded here. Version 31.0 of the O*NET database was used and the required data files are available here as an archive (data downloaded from O*NET on 17/8/2026).

Background

In 2025 I worked as an adviser for a project mapping alternative occupational transition pathways in India to understand longer-term structural changes in the labor market and the impacts of AI. Aside from being an interesting piece of work, it gave me a chance to dig deeper into the literature exploring how to measure human capital and model labor market dynamics.

For those unfamiliar with the term human capital, it’s essentially a catch-all category for characteristics that influence a worker’s productivity, such as skills, expertise and emotional intelligence. Unsurprisingly, defining and measuring this is somewhat of an obsession for economists: both to clarify exactly what it means and as there’s solid evidence that it influences social and economic outcomes.

Measuring Human Capital

As enthralling as human capital’s definition might be, I’ll just note that until recently it was conceptualized as some combination of education and experience.¹ This isn’t to say that economists believed it was this simple, but just that this was the empirical shorthand that was often used by researchers.

Enter Autor, Levy and Murnane (2001)² who posited that occupations and the demand for skills could be better conceptualized through tasks, such as moving an object, executing a calculation, communicating a piece of information and resolving a discrepancy. For instance, a truck driver might need to operate a vehicle, monitor its condition, plan a route, keep records and deal with customers at delivery. This meant measuring human capital based on what a worker can do rather than the credentials and experience they hold. Although a lot of ground has moved since then³, the basic idea is what makes occupations comparable at all: if a job is a collection of tasks, two jobs built from similar tasks are jobs a worker could more easily move between.

A Primer on the O*NET

The US Occupational Information Network (O*NET) (and its predecessor) provides definitions and ratings across a set of characteristics for more than 800 occupations, mapped to the US Standard Occupational Classification (SOC):

Every occupation requires a different mix of knowledge, skills, and abilities, and is performed using a variety of activities and tasks. These distinguishing characteristics, or “descriptors”, of an occupation are collected, codified, and described by the O*NET Content Model, This hierarchical model starts with six domains (or categories), describing the day-to-day aspects of the job and the qualifications and interests of the typical worker.

Abilities describe the attributes that relate to who a worker is and how they work; while skills and knowledge entail what a worker needs to know in order to perform tasks required in an occupation (see here for more).

The figure below presents an example of abilities ascribed to Lawyers (23-1011.00). Ratings are provided for each occupation. Speech Clarity, for instance, is provided a rating for both Clergy and Lawyers. Each characteristic is also divided into smaller categories and sub-categories based on common themes: Oral Expression sits under Verbal Abilities, which sits under Cognitive Abilities. This groups more comparable statistics with one another.

Each characteristic is then rated based on its level and importance. Importance is meant to measure how critical a characteristic is to a job, whereas the level signifies the level of proficiency required / complexity of the task. For instance, the skill of speaking is important for both lawyers and paralegals, but lawyers are expected to have a higher Level of speaking skill compared to a paralegal (see here):

Comparing Occupations

By measuring a detailed set of standardized characteristics, the O*NET makes it possible to and ask more interesting questions about human capital and the labor market. Understanding how similar jobs are to one another is one such example. Workers should find it easier to transition across occupations that require a similar set of skills, abilities and knowledge, which can be a useful thing to know if it’s expected that some industries will face structural changes or be disrupted by new technology, such as artificial intelligence.

This is the point of similarity scores: to provide a proxy for how similar two occupations are and the ease of transitioning between them. Like Bratanova et al. (2026), we’ll calculate similarity scores by comparing abilities, skills and knowledge. For the sake of brevity, we won’t consider the viability of transition paths, which will depend on wage differentials, employment demand and other costs associated with making the move. But, Bratanova et al. (2026) did and you should too if you’re intending to use similarity scores in the real world.

Methodological Differences

One of the motivations for writing this post (and arguing with Claude about the code) was to improve my understanding of the O*NET and build a baseline for future analysis. It’s therefore intended to demonstrate Bratanova et al.’s (2026) approach, rather than precisely replicate their results. The methodology applied here also differs in three important ways:

• We use a later version of the O*NET database.
• Our ANZSCO to SOC mapping differ to theirs. This is partially due to updates made to the classification standards and as our SOC to ANZSCO mapping is based on making literal joins across the published correspondences (see here).
• Bratanova et al. (2026) assign one O*NET occupation to each ANZSCO group, where ours sometimes assigns several. That means averaging characteristics across occupations before comparing them, resulting in our scores describing a typical job in each group rather than a single representative occupation.

Project Setup

The code below loads the required packages and defines some the assumptions used throughout the analysis.

library(tidyverse)
library(janitor)
library(readxl)
library(ggridges)

# Where the data lives.
ref_dir_data <- file.path(".", "Data")

# The O*NET element files. The NAME of each entry becomes the element type
# label in the combined table.
ref_files_onet <- c(
  abilities = "Abilities.xlsx",
  skills_essential    = "Essential Skills.xlsx",
  skills_transferable = "Transferable Skills.xlsx",
  knowledge = "Knowledge.xlsx"
)

# The correspondence table mapping O*NET (US SOC) occupations onto ANZSCO.
ref_file_cross <- "260821 - crosswalk_for_onet.csv"

# The columns that define the occupation groups scores are reported for.
ref_col_group_code  <- "anzsco_code"
ref_col_group_label <- "label_4digit"

# The occupation being scored FROM, as it appears in ref_col_group_label.
# The focus of Bratanova et al. (2026) was truck drivers, which is why they are
# selected here.
ref_occ_origin <- "Truck Driver (General)"

# O*NET rating scale bounds, used to put both ratings on a 0-100 scale.
# Importance is collected on a 1-5 scale, level on a 0-7 scale.
ref_scale_level_max <- 7
ref_scale_imp_min   <- 1
ref_scale_imp_max   <- 5

# Whether the origin occupation is compared against itself. TRUE is a helpful
# check that the scores make sense: the origin should score 100.
# It also sets to maximum similarity score (100%) to movements to the same 
# occupational group
ref_incl_self_pair <- TRUE

# How similar two occupations must be for a transition to be technically
# viable, ignoring education, geography and wages. Also from Bratanova et al.
ref_sim_threshold <- 70

#chart colors
ref_col_primary <- "#1E298D"  # dark blue
ref_col_accent  <- "#26CDDA"  # cyan
ref_col_grid    <- "#E5E7EB"

Import the data

The following functions are used to import, label and combine the O*NET files into a single table.

Note: The code below combines transferable and essential skills into a single group so they are treated identically in the analysis. This approach was retained for the sake of simplicity, but tests indicated keeping each group separate had minimal impact on the final similarity scores.

dta_onet_raw <- imap(ref_files_onet, \(ref_file, ref_type) {
  read_excel(file.path(ref_dir_data, ref_file)) |>
    clean_names() |>
    mutate(onet_element_subtype = ref_type,
           onet_element_type    = str_remove(ref_type, "_.*"))
}) |>
  list_rbind() |>
  rename(soc_code  = o_net_soc_code,
         soc_label = title)

# The correspondence table
lkp_cross <- read_csv(file.path(ref_dir_data, ref_file_cross),
                      show_col_types = FALSE) |>
  clean_names() |>
  mutate(across(all_of(ref_col_group_code), as.character)) |> 
  rename(soc_code=soc,
         anzsco_label=label_4digit)

Exploratory Analysis

Checking Correspondence Coverage

Because the point of this post is show how the O*NET can be used to compare different occupations, we won’t delve too deeply into the coverage, representativeness or relevance of the results, but all of these considerations do matter. In our case, mapping SOC to ANZSCO results in a little over ten percent of occupations being dropped, which might change the number of viable transition pathways and shift the rated similarity of the remaining occupations. How much that matters largely depends on whether the dropped jobs look anything like the remaining ninety percent, which is a good question, but one for another post.

#number of unique O*NET code
length(unique(dta_onet_raw$soc_code))

#how many of the O*NET Occupations exist in the correspondence table?
#in the ANZSCO case, 89 percent of the O*NET data can be matched
#we won't focus too much on the unmatched occupations here  

table(dta_onet_raw$onet_element_type, dta_onet_raw$soc_code %in% lkp_cross$soc_code) |> prop.table(margin=1) |> round_half_up(2)

Examining Element Types

Our analysis focuses on three worker-orientated characteristics recorded by the O*NET: skills, abilities and knowledge. Each of these characteristics is divided into specific elements and rated based on their level and importance. The code below presents and example of this by counting the number of values by element type (abilities, skills and knowledge) and element name (oral expression, mathematics, critical thinking etc).

Values represent either the level or importance of a characteristic. For instance, for Chief Executives Explosive Strength has an importance of 1 and expected level of 0, whereas both are rated close to 4 for Athletes and Sports Competitors.

# Should be 35 skills, 52 abilities and 33 knowledge areas.
sum_element_types <- dta_onet_raw |>
  group_by(scale_name,onet_element_type, element_name) |> 
  summarise(nmb_elements = sum(!is.na(data_value))) |> 
  pivot_wider(names_from = scale_name, values_from=nmb_elements)

sum_element_types

Estimating Similarity Scores

An Overview of the Approach

The idea is simple enough: if two occupations are similar they’ll require workers with similar attributes. To make this comparison takes four main steps:

1. Each ANZSCO group is assigned a set of occupations from the O*NET based on our crosswalk / correspondence and their ratings are averaged to represent a typical job in each group.
2. Ratings are placed on a common scale (0 to 100) so they can be compared and aggregated. O*NET rates levels from 0 to 7 and importance from 1 to 5.
3. The origin occupation (truck drivers) are compared against every other occupation one characteristic at a time. For each characteristic, the absolute difference in level and in importance is taken to estimate the skill gap. The larger the gap, the bigger the difference between jobs.
4. Gaps are averaged across characteristics and turned into a score from 0 to 100. The closer the score is to 100 the more similar the occupations being compared are to one another.

The paper defines, for origin occupation t and destination j:

The code below applies the defined calculations to the data, but average the gaps rather than summing them. This is essentially the same thing, just easier to interpret. It’s also worth noting that because we’ve chosen to retain comparisons between the same occupation (e.g. Truck Driving to Truck Driving), the minimum skill gap is anchored to zero, making the maximum similarity score based on transitioning to the same occupation.

Preparing the O*NET Data

The code below joins the SOC to ANZSCO correspondence table to the O*NET dataframe. The use of a many-to-many join reflects the fact that the same occupation from the SOC can apply to more than one ANZSCO grouping (and vice-versa). For instance, the SOC occupation Light Truck Drivers is allocated to four ANZSCO unit groups: Ambulance Officer, Courier, Chauffeur and Delivery Driver. While ANZSCO’s Truck Driver (General) is matched with more than one occupations from the O*NET: Heavy and Tractor-Trailer Truck Drivers and Industrial Truck and Tractor Operators.

The practical implications of matching the O*NET data to the ANZSCO like this is that similarity scores are based on the average characteristics within an ANZSCO unit group, which differs from Bratanova et al. (2026), who assigned one O*NET occupation per ANZSCO unit group.

# Step 1. Average the O*NET ratings onto the occupation groups. The join is
# many-to-many to reflect O*NET occupations can exist in more than one ANZSCO unit # group. 

#join cross table to O*NET dataframe: 
#(drops soc_label from lkp_cross to avoid it being duplicated) 
dta_onet_groups <-inner_join(dta_onet_raw, 
                             lkp_cross |> 
                               select(-soc_label), 
                             by = "soc_code",
             relationship = "many-to-many") 

#group O*NET data by occupations for each characteristic / scale and calculate the average score
dta_onet_groups<-dta_onet_groups |> 
group_by(anzsco_code,anzsco_label, onet_element_type, element_name, scale_name) |> 
  summarise(
    value = mean(data_value))

#widen the dataframe so importance and level averages are in separate columns:
dta_onet_groups<-dta_onet_groups |> 
  ungroup() |> 
  mutate(scale_name = str_to_lower(scale_name)) |>
  pivot_wider(names_from = scale_name, values_from = value)

Demonstration: Comparing Skill Gaps

The code snippet below demonstrates the comparison logic by comparing the level and importance of a small set of skills for truck drivers, bus drivers and finance managers.

Notice the gaps in level and importance for Operation and Control: small between truck drivers and bus drivers, but comparatively large when they’re compared to finance managers. On the other hand, Management of Financial Resources is rated highly for finance managers when compared to truck and bus drivers. Similarity scores are based on these gaps, with the idea being that occupations with smaller gaps tend to be more similar to one another than those with large gaps.

ref_occ_demo <- c("Bus Driver","Finance Manager")

#five skill elements, held fixed across the worked examples below:
ref_elements_demo <- dta_onet_groups |>
  filter(onet_element_type == "skills", 
         element_name == "Management of Financial Resources"|
         element_name== "Operation and Control") |>
  distinct(element_name) |>
  arrange(element_name) |>
  slice_head(n = 10) |>
  pull(element_name)

#the averaged O*NET ratings for the two occupations, still on their own scales:
dta_onet_groups |>
  filter(anzsco_label %in% c(ref_occ_origin, ref_occ_demo),
         element_name %in% ref_elements_demo) |>
  select(anzsco_label, element_name, importance, level) |>
  arrange(element_name, anzsco_label) |>
  mutate(across(c(importance, level), \(x) round_half_up(x, 1)))

Occupation	Element	Importance	Level
Bus Driver	Management of Financial Resources	1.5	0.6
Finance Manager	Management of Financial Resources	3.6	4.3
Truck Driver (General)	Management of Financial Resources	1.7	1.1
Bus Driver	Operation and Control	3.4	2.9
Finance Manager	Operation and Control	1.1	0.1
Truck Driver (General)	Operation and Control	3.8	3.3

Put Level and Importance Scores on a Common Scale

The code below put the level and importance scores on a common scale (0 to 100) based on score ranges prescribed by the O*NET (0 to 7 for level and 1 to 5 for importance):

# Step 2. Put importance and level on a common 0-100 scale.
dta_onet_scaled <- dta_onet_groups |>
  mutate(
    level      = 100 * level / ref_scale_level_max,
    importance = 100 * (importance - ref_scale_imp_min) /
                       (ref_scale_imp_max - ref_scale_imp_min))

Demonstration: Scaling Scores

The code below demonstrates how scaling works using the same occupations pairs presented earlier. If you’re comfortable with code this won’t be particularly surprising:

dta_onet_groups |>
  filter(anzsco_label %in% c(ref_occ_origin, ref_occ_demo),
         element_name %in% ref_elements_demo) |>
  select(anzsco_label, element_name,
         importance_raw = importance, level_raw = level) |>
  left_join(dta_onet_scaled |>
              select(anzsco_label, element_name,
                     importance_0_100 = importance, level_0_100 = level),
            by = join_by(anzsco_label, element_name)) |>
  arrange(element_name, anzsco_label) |>
  mutate(across(where(is.numeric), \(x) round_half_up(x, 1)))

Occupation	Element	Importance (raw)	Level (raw)	Importance (0–100)	Level (0–100)
Bus Driver	Management of Financial Resources	1.5	0.6	12.5	8.9
Finance Manager	Management of Financial Resources	3.6	4.3	64.0	61.6
Truck Driver (General)	Management of Financial Resources	1.7	1.1	17.1	16.1
Bus Driver	Operation and Control	3.4	2.9	59.5	41.0
Finance Manager	Operation and Control	1.1	0.1	3.0	1.7
Truck Driver (General)	Operation and Control	3.8	3.3	70.4	47.3

Creating Occupation Pairs for Comparison

To calculate the similarity score for every possible destination occupation a truck driver might move to, the code below applies a many-to-many join based on the element type and name. This places the same abilities, skill and knowledge next to each other for every destination occupation a truck driver might transition so the gaps can be calculated.

# Step 3. Pair the origin occupation against every destination, element by
# element. a is where you are now, b is where you might go.

#create origin and destination dataframes prior to joining
#origin dataframe includes only origin occupation
dta_onet_origin <- dta_onet_scaled |>
  filter(anzsco_label == ref_occ_origin) |>
  rename_with(\(x) paste0(x, "_a"), !c(onet_element_type, element_name))

#now for origin occupations: 
dta_onet_dest <- dta_onet_scaled |>
  rename_with(\(x) paste0(x, "_b"), !c(onet_element_type, element_name))

#merge both dataframes to create origin-destination occupation pairs: 
dta_onet_pairs <- dta_onet_origin |>
  inner_join(dta_onet_dest, by = join_by(onet_element_type, element_name),
             relationship = "many-to-many") |>
  mutate(
    gap_importance     = abs(importance_b - importance_a),
    gap_level          = abs(level_b - level_a),
    surplus_importance = pmax(importance_a - importance_b, 0),
    surplus_level      = pmax(level_a - level_b, 0)
  )

Demonstration: Skill Gaps

The code below continues the earlier example to illustrate what this looks like in practice. Notice that the skill gaps are smaller between truck drivers and bus drivers for both characteristics pointing to occupation bus driver being comparatively similar to truck driving than finance manager.

dta_onet_pairs |>
  filter(anzsco_label_b %in% ref_occ_demo,
         element_name %in% ref_elements_demo) |>
  select(anzsco_label_a,anzsco_label_b, element_name,
         level_a, level_b, gap_level,
         importance_a, importance_b, gap_importance) |>
  arrange(anzsco_label_b,element_name) |>
  mutate(across(where(is.numeric), \(x) round_half_up(x, 1)))

From	To	Element	Level (from)	Level (to)	Level gap
Truck Driver (General)	Bus Driver	Management of Financial Resources	16.1	8.9	7.2
Truck Driver (General)	Bus Driver	Operation and Control	47.3	41.0	6.3
Truck Driver (General)	Finance Manager	Management of Financial Resources	16.1	61.6	45.4
Truck Driver (General)	Finance Manager	Operation and Control	47.3	1.7	45.6

Calculating Occupation Pair Similarity

To create a generalized similarity score, the code below estimates the average gaps for each occupation-pair and translates this into a similarity score ranging from 0 to 100 percent, with scores closer to 100 percent indicating the occupational pair are on average more similar to one another.

# Step 4. Average the element-by-element gaps up to one row per occupation pair.
sum_onet_gaps <- dta_onet_pairs |>
  group_by(anzsco_code_a, anzsco_label_a,
           anzsco_code_b, anzsco_label_b) |>
  #calculate the average gaps and surpluses by occupation:
  summarise(
    gap_importance_mean     = mean(gap_importance),
    gap_level_mean          = mean(gap_level),
    surplus_importance_mean = mean(surplus_importance),
    surplus_level_mean      = mean(surplus_level),
    nmb_elements            = n()) |>
  #calculate the average gap and surplus across level and importance:
  mutate(
    gap_both_mean     = (gap_importance_mean + gap_level_mean) / 2,
    surplus_both_mean = (surplus_importance_mean + surplus_level_mean) / 2
  ) |>
  ungroup()

#tests whether each pairing is compared over the same number of elements:
n_distinct(sum_onet_gaps$nmb_elements) == 1

# Dropping the self-comparison has to happen before the scores are rescaled,
# because it changes the minimum:
if (!ref_incl_self_pair) {
  sum_onet_gaps <- filter(sum_onet_gaps, anzsco_code_a != anzsco_code_b)
}

# Turn a gap into a 0-100 score: the smallest gap becomes 100, the largest 0.
fnc_sim_from_gap <- function(ref_gap) {
  100 * (1 - (ref_gap - min(ref_gap)) / (max(ref_gap) - min(ref_gap)))
}

rlt_similarity <- sum_onet_gaps |>
  mutate(
    sim_both       = fnc_sim_from_gap(gap_both_mean),
    sim_level      = fnc_sim_from_gap(gap_level_mean),
    sim_importance = fnc_sim_from_gap(gap_importance_mean)
  ) |>
  arrange(desc(sim_both))

Demonstration: Calculating Similarity Scores

The code below shows how average skill gaps become similarity scores. Truck drivers score 100 against themselves, as gap_mean is zero. Bus drivers scoring higher than finance managers points to the occupation having similar characteristics to truck drivers, which should make the moving between the jobs simpler.

#the two anchors the rescaling uses:
sum_demo_anchors <- sum_onet_gaps |>
  summarise(gap_min   = min(gap_both_mean),
            gap_max   = max(gap_both_mean),
            occ_min   = anzsco_label_b[which.min(gap_both_mean)],
            occ_max   = anzsco_label_b[which.max(gap_both_mean)])

sum_demo_anchors |>
  mutate(across(where(is.numeric), \(x) round(x, 2)))


#add Truck Drivers to show like-like comparison 
ref_occ_demo <- c("Finance Manager","Bus Driver","Truck Driver (General)")

#and where the worked example sits between them:
sum_onet_gaps |>
  filter(anzsco_label_b %in% ref_occ_demo) |>
  transmute(anzsco_label_b,
            gap_both_mean,
            gap_min       = sum_demo_anchors$gap_min,
            gap_max       = sum_demo_anchors$gap_max,
            share_of_range = (gap_both_mean - gap_min) / (gap_max - gap_min),
            sim_both      = 100 * (1 - share_of_range)) |>
  mutate(across(where(is.numeric), \(x) round_half_up(x, 2))) |> 
  arrange(desc(anzsco_label_b))

Destination	Mean gap	Smallest gap	Largest gap	Share of range	Similarity score
Truck Driver (General)	0.00	0	27.4	0.00	100.0
Bus Driver	7.22	0	27.4	0.26	73.7
Finance Manager	25.30	0	27.4	0.92	7.8

Demonstration: Skill Gap Distribution

Because the scale is scaled based on the smallest and largest gap, each similarity score represents the relative similarity rather than some absolute quantity. The chart below demonstrates this idea using the same three examples:

#positions and labels from one table, so they can't fall out of step when the
#origin is dropped by ref_incl_self_pair:
dta_gap_demo <- sum_onet_gaps |>
  filter(anzsco_label_b %in% ref_occ_demo) |>
  select(anzsco_label_b, gap_both_mean)

sum_onet_gaps |>
  ggplot(aes(x = gap_both_mean)) +
  geom_histogram(bins = 60, fill = ref_col_grid) +
  geom_vline(data = dta_gap_demo, aes(xintercept = gap_both_mean),
             colour = ref_col_primary, linewidth = 1) +
  geom_text(data = dta_gap_demo,
            aes(x = gap_both_mean, y = Inf, label = anzsco_label_b),
            colour = ref_col_primary, hjust = -0.05, vjust = 1.6, size = 3.4) +
  theme_minimal(base_size = 11) +
  theme(panel.grid.minor = element_blank(),
        plot.title.position = "plot") +
  labs(title = paste0("Average skill gap from ", ref_occ_origin,
                      " to every other unit group"),
       subtitle = "Scores run from 100 at the smallest gap to 0 at the largest",
       x = "Average gap across level and importance (0-100)",
       y = "Number of unit groups")

Examining Results

Hard-Coding Published Results

The code below hardcodes Table 2 from Bratanova et al. (2026).

Note: ANZSCO codes were matched approximately based on job titles, not official correspondences.

# Table 2, with each published occupation matched by hand to its closest ANZSCO
# 2024 unit group. NA means no close equivalent was identified. 
rlt_sim_published <- tibble::tribble(
  ~anzsco_code_published, ~anzsco_title_published,                                                ~sim_published, ~anzsco_code, ~anzsco_label,
  "7121", "Crane, Hoist and Lift Operators",                                78, "7121", "Crane, Hoist or Lift Operator",
  "7321", "Delivery Drivers",                                               76, "7321", "Delivery Driver",
  "7212", "Earthmoving Plant Operators",                                    75, "7212", "Earthmoving Plant Operator (General)",
  "7312", "Bus and Coach Drivers",                                          75, "7312", "Bus Driver",
  "7313", "Train and Tram Drivers",                                         75, "7313", "Train Driver",
  "8216", "Railway Track Workers",                                          74, NA,     NA,
  "7122", "Drillers, Miners and Shot Firers",                               74, "7122", "Driller",
  "8219", "Other Construction and Mining Labourers",                        73, NA,     NA,
  "8992", "Deck and Fishing Hands",                                         73, "8992", "Deck Hand",
  "7211", "Agricultural, Forestry and Horticultural Plant Operators",       73, NA,     NA,
  "7213", "Forklift Drivers",                                               72, "7213", "Forklift Driver",
  "7219", "Other Mobile Plant Operators",                                   72, "7219", "Aircraft Baggage Handler and Airline Ground Crew",
  "8215", "Paving and Surfacing Labourers",                                 72, "8215", "Paving and Surfacing Labourer",
  "7123", "Engineering Production Workers",                                 72, "7123", "Engineering Production Worker",
  "8321", "Packers",                                                        71, NA,     NA,
  "8391", "Metal Engineering Process Workers",                              71, "8391", "Metal Engineering Process Worker",
  "8413", "Forestry and Logging Workers",                                   70, NA,     NA
)

Join Published Results to Ours

To allow side-by-side comparison, the code below joins the published similarity scores with our own. Because the ANZSCO groupings differ, some results have been dropped.

# Join this run's scores onto the published table using the latest ANZSCO codes 
dta_comparison_all <- rlt_sim_published |>
  left_join(
    rlt_similarity |>
      select(anzsco_code = anzsco_code_b, sim_reproduced = sim_both),
    by = join_by(anzsco_code),
    relationship = "one-to-one"
  )

# Published occupations that drop out, and why. A miss is a coverage result,
# not a broken script, but it has to be visible or the comparison flatters
# itself.
chk_missing <- dta_comparison_all |>
  filter(is.na(sim_reproduced)) |>
  transmute(anzsco_code, anzsco_title_published,
            reason = if_else(is.na(anzsco_code),
                             "no ANZSCO 2024 match identified",
                             "matched but not scored by this run"))
# five occupations dropped 
chk_missing

#calculate difference between published and reproduced scores
dta_comparison <- dta_comparison_all |>
  filter(!is.na(sim_reproduced)) |>
  mutate(sim_difference = sim_reproduced - sim_published)

Produce Figure Comparing Results

The figure below compares our results with those reported by Bratanova et al. (2026). Where occupations could be matched the scores align well, though a handful sit up to fifteen points above the published values. The likely reasons for these differences are set out below. But, this felt like a pleasant surprise given differences in our source data, correspondences and classification system.

The code below produces the figure above comparing our similarity scores with the published paper:

plt_comparison <- dta_comparison |>
  mutate(anzsco_title_published = fct_reorder(anzsco_title_published, sim_published)) |>
  pivot_longer(c(sim_published, sim_reproduced),
               names_to = "source", values_to = "sim_score") |>
  mutate(source = if_else(source == "sim_published",
                          "Published", "Reproduced here"))

#label the two points on the widest pair, where there is room for text:
dta_plot_labels <- plt_comparison |>
  mutate(gap = max(sim_score) - min(sim_score), .by = anzsco_title_published) |>
  filter(gap == max(gap))

plt_comparison <- plt_comparison |>
  ggplot(aes(x = sim_score, y = anzsco_title_published)) +
  geom_line(aes(group = anzsco_title_published),
            colour = ref_col_grid, linewidth = 2, lineend = "round") +
  geom_point(aes(colour = source), size = 2.8) +
  geom_text(data = dta_plot_labels,
            aes(label = source, colour = source),
            hjust = 0.5, vjust = -1.4, size = 3.2, fontface = "bold") +
  scale_colour_manual(values = c("Published"       = ref_col_primary,
                                 "Reproduced here" = ref_col_accent),
                      guide = "none") +
  scale_x_continuous(limits = c(65, 100),
                     breaks = seq(70, 100, 10),
                     expand = expansion(mult = c(0.01, 0.02))) +
  theme_classic(base_size = 11) +
  theme(
    plot.title.position = "plot",
    plot.title    = element_text(face = "bold", size = rel(1.15)),
    plot.caption.position = "plot",
    plot.caption  = element_text(colour = "grey45", hjust = 0),
    axis.title.x  = element_text(margin = margin(t = 8), colour = "grey30"),
    axis.text.y   = element_text(colour = "grey20"),
    axis.ticks    = element_blank()
  ) +
  labs(
    title = str_wrap(paste0("Reproduced scores run above the published ones for ",
                            ref_occ_origin), width = 80),
    x = "Skill similarity score",
    y = NULL,
    caption = "Published scores based on: Bratanova et al. (2026), Transport Policy 176, Table 2."
  )

plt_comparison

Similarity Score Distribution by ANZSCO Major Groups

The figure below presents the distribution of similarity scores by each ANZSCO major group. Overall, occupations in the Professionals and Managers grouping have fewer jobs with similar skills to truck drivers when compared with Technicians and Trades Workers, Laborers and Machinery Operators and Drivers.

The code below produces a ridgeline graph of similarity scores by ANZSCO major groups (see here for group codes).

lkp_anzsco_major <- tibble::tribble(
  ~anzsco_major, ~major_label,
  "1", "Managers",
  "2", "Professionals",
  "3", "Technicians and Trades Workers",
  "4", "Community and Personal Service Workers",
  "5", "Clerical and Administrative Workers",
  "6", "Sales Workers",
  "7", "Machinery Operators and Drivers",
  "8", "Labourers"
)

#one distribution per ANZSCO major group: how similar every occupation in that
#group is to truck drivers. Most similar group at the bottom.
dta_similarity_distributions <- rlt_similarity |>
  mutate(anzsco_major = str_sub(anzsco_code_b, 1, 1)) |>
  left_join(lkp_anzsco_major, by = join_by(anzsco_major)) |>
  mutate(major_label = paste0(major_label, " (", anzsco_major, ")")) |>
  mutate(sim_median = median(sim_both), .by = major_label) |>
  mutate(major_label = fct_reorder(major_label, -sim_median))

plt_sim_score_distribution<- ggplot(dta_similarity_distributions,
       aes(x = sim_both, y = major_label, fill = sim_median)) +
  geom_density_ridges(scale = 2.4, colour = "white", linewidth = 0.4,
                      rel_min_height = 0.005) +
  scale_fill_gradient(low = ref_col_primary, high = ref_col_accent) +
  scale_x_continuous(expand = expansion(mult = c(0.02, 0.02))) +
  scale_y_discrete(expand = expansion(add = c(0.2, 1.9))) +
  guides(fill = "none") +
  theme_minimal(base_size = 11) +
  theme(
    panel.grid.major.y  = element_blank(),
    panel.grid.minor    = element_blank(),
    axis.text.y         = element_text(vjust = 0, colour = "grey20"),
    axis.title.x        = element_text(margin = margin(t = 8), colour = "grey30"),
    plot.title.position = "plot",
    plot.title    = element_text(face = "bold"),
    plot.subtitle = element_text(colour = "grey30", margin = margin(b = 14))
  ) +
  labs(
    title = "Similarity Score Distribution by ANZSCO Group", 
  subtitle = str_wrap(paste0(
      "Each ridge presents the distribution of similarity scores by ANZSCO major group. Distributions further to the right contain occupations with more skills, abilities and knowledge in common with truck drivers."),
      width = 120),
    x = "Skill Similarity Score",
    y = NULL)

plt_sim_score_distribution

Summing Up

Although our results don’t precisely match those published by Bratanova et al. (2026), they’re closer than might be expected given differences in the source data and approach for mapping O*NET data to the ANZSCO. The post has also served the purpose that it was meant to: providing a reproducible demonstration of how similarity scores work.

Having said this, I did spend quite a bit of time trying to find the source the discrepancies by testing how scores changed when altering assumptions and data (e.g. crosswalks and O*NET data). This indicated that the crosswalk was the culprit, with scores being higher across a particular collection of occupations for two reasons:

• Breadth: because our correspondence takes a journey through the ISCO-08 occupational definitions, our mapping assigns a larger number of similar occupations from the O*NET to the same ANZSCO groups. This results in similar O*NET occupations being assigned to adjacent unit groups (attested by the outliers being in adjacent ANZSCO groups and being assigned similar occupations from the O*NET).
• Overlap: As a result of changes in the ANZSCO and the ISCO-08 issue noted above, many of the ANZSCO unit groups have been assigned the same occupations. For instance, both Truck Driver (General) and Forklift Driver are assigned Industrial Truck and Tractor Operators from the O*NET, with the overlap boosting the similarity score. Sometimes there is also overlap between destinations, for instance Other Mobile Plant Operators and Earthmoving Plant Operator (General) are assigned an identical set of six O*NET occupations, which is why they have identical scores (they’re identical profiles).

Just how much the crosswalk matters will come as no surprise to anyone who has worked with one before. It’s also why I built the naive correspondence used here, as I’m hoping my rough approach will encourage others to build something better. However, that’s an aside. The point of this post was to demonstrate how the O*NET can be used to calculate similarity scores for identifying potential transition paths across the labor market, which I’ll give myself an 11/10 for.

But, it’s worth remembering that similarity scores can tell us only so much. They say nothing about whether the pathway is real. They don’t tell you how far a worker might have to travel (or move), or whether the differences in wage rates and working conditions make it worthwhile. And they say nothing about the personal costs either, such as the professional relationships that are left behind, strained social connections or the real work of needing to reshaping an identity that may be intimately connected to what they do.

Yet, similarity scores have real value. Firstly, there’s evidence that these score hold weight in the real world, with research indicating that measures of similarity do predict actual movements across the labor market, including the Bratanova paper we’ve drawn on here.. They also narrow the field for designing better policy: knowing the range of viable transition paths allows policy to be designed to target workers most likely to benefit from support.

Finally, the methodology (and code) are relatively simple to apply to other contexts, provided you can develop a mapping from your own national classification codes to the SOC. Whether the O*NET provides an accurate enough proxy for jobs outside the US is a question for another post. Until then, I’ll take comfort in the thought that somewhere a new version of Claude is being trained to argue with me about it.

How AI was used to write this post

The first draft of the analysis pipeline was written by me based on the code and data shared by Bratanova et al. (2026). Once I’d reproduced their results, I had AI help me generalize the pipeline to make it easier to adapt to occupational definitions outside the ANZSCO. The text is almost entirely mine, with AI mainly used for tweaking how ideas are communicated and converting tables so they could be presented in this post.

1. Becker, G.S., 1975. Investment in human capital: effects on earnings. In Human Capital: A Theoretical and Empirical Analysis, with Special Reference to Education, Second Edition (pp. 13-44). NBER. ︎
2. Autor, D., Levy, F. and Murnane, R.J., 2001. The skill content of recent technological change: an empirical exploration. https://www.nber.org/system/files/working_papers/w8337/w8337.pdf ︎
3. Acemoglu, D. and Autor, D.H., 2011. Chapter 12-skills, tasks and technologies: Implications for employment and earnings (d. card & o. ashenfelter, eds.). Elsevier. https://doi. org/10.1016/S0169-7218 (11), pp.02410-5. https://economics.mit.edu/sites/default/files/publications/Skills%2C%20Tasks%20and%20Technologies%20- %20Implications%20for%20.pdf ︎

The post Identifying skills-based occupational transition pathways with the O*NET appeared first on Giles.