1) Maps

This map provides an intuitive understanding of how to apply the line-up protocol to a real-world example. Richard uses data from the GLES (German Longitdunal Election Survey) as an example to analyze the interviewer selection effects. These biases arise if interviewers selectively contact certain households and fail to reach to others. Reasons might be that researchers try to avoid less comfortable areas.

As a first step, we need to read in the required packages as well as the data and code the interviewer behavior by color.

# Read all required packages
library(maps)
library(mapdata)
library(RColorBrewer)

# Read data
data <- readRDS("sub_data.rds")

# Code interviewer behavior by color
data$col <-
  ifelse(data$status == "No Contact", "maroon3", "darkolivegreen2")

Following the line-up protocol described above, we seek to answer the question if there is a visual pattern. Our null hypothesis assumes that there is no visual pattern. To generate the null dataset, we randomly permute one variable column and plot the data.

# Generate random plot placement
placement <- sample((1:20), 20)
layout(matrix(placement, 4, 5))

# Generate 19 null plots
par(mar = c(.01, .01, .01, .01), oma = c(0, 0, 0, 0))
for(i in 1:19) {
  # Randomize the order
  random <- sample(c(1:15591), 15591)
  # Plot
  map(
    # Refer to dataset
    database = "worldHires",
    fill = F,
    col = "darkgrey",
    # Range of x-axis
    xlim = c(6, 15),
    # Range of y-axis
    ylim = c(47.3, 55)
    ) 
  points(
    # Refer to data
    data$g_lon,
    data$g_lat,
    cex = .1,
    # Type of plotting symbol
    pch = 19,
    col = data$col[random]
    )
}

We then proceed and add the true data to the null datasets.

# Add the true plot
map(
  database = "worldHires",
  fill = F,
  col = "darkgrey",
  # Range of x-axis
  xlim = c(6, 15),
  # Range of y-axis
  ylim = c(47.3, 55)
  ) 
points(
  # Refer to data
  data$g_lon,
  data$g_lat,
  cex = .1,
  # Type of plotting symbol
  pch = 19,
  col = data$col
  )

# Reveal the true plot
box(col = "red", # Draw a box in red
    lty = 2, # Defines line type
    lwd = 2) # Defines line width
    which(placement == 20) # Defines the place of the box

Using the code above, we receive twenty maps from Germany. In a last step, we ask if these plots are substantially different from one another. If yes, can you tell which one is the odd-one-out? Just wait for a few seconds to let the image reveal the answer.

2) Scatter plot

Mimicking the approach for the maps, we proceed in a similar way with scatter plots.

Assume we have two variables and want to plot their correlation with a scatter plot. To compare if their relation is random, we can make use of visual inference. To do so, we first need to load all required packages and read in the data.

# Read required package
library(foreign) # Necessary to load datasets in other formats (such as .dta)

# Read the data
slop <- read.dta("slop_2009_agg_example.dta")

We then proceed and place randomly 20 plots within a 4x5 grid cells.

# Generate a random plot placement
placement <- sample((1:20), 20)
layout(matrix(placement, 4, 5))

We want to position 19 out of 20 random plots and leave one grid cell empty for the “true” plot.

# Plot 19 null plots
par(mar = c(.1, .1, .1, .1))
for(i in 1:19) {
  # Plot random scatter plots of the data
  random <- sample(c(1:dim(slop)[1]), dim(slop)[1])
  plot(slop$mkath[random],
  slop$cdu,
  axes = F,
  ann = F,
  cex = .4)
  # Plot a box with grey lines
  box(bty = "l", 
  col = "grey")
}

As we can see, we get a 4x5 grid cell with 19 randomly assigned scatter plots and one empty cell. We now proceed and fill this empty cell with the “true” data and plot a box around it.

# Add true plot
plot(slop$mkath,
     slop$cdu,
     axes = F,
     ann = F,
     cex = .4)
box(bty = "l", # Plot a box with grey lines
    col = "grey")

# Reveal true plot
box(col = "red", # Plot a box with red dashed lines
    lty = 2,
    lwd = 2)
which(placement == 20) # Define the position of the box

We can even go one step further by adding an abline to the plots. To do this, we need to include the following line of code:

abline(lm(slop$cdu ~ slop$mkath[random]))

# Generate a random plot placement
placement <- sample((1:20), 20)
layout(matrix(placement, 4, 5))

# Plot 19 null plots
par(mar = c(.1, .1, .1, .1))
for(i in 1:19) {
  # Plot random scatter plots of the data
  random <- sample(c(1:dim(slop)[1]), dim(slop)[1])
  plot(slop$mkath[random],
  slop$cdu,
  axes = F,
  ann = F,
  cex = .4)
  # Add the abline to the plots
  abline(lm(slop$cdu ~ slop$mkath[random]))
  # Plot a box with grey lines
  box(bty = "l",
  col = "grey")
}
# Add true plot
plot(slop$mkath,
     slop$cdu,
     axes = F,
     ann = F,
     cex = .4)
abline(lm(slop$cdu ~ slop$mkath)) # Add the abline to the plot
box(bty = "l", # Plot a box with grey lines
    col = "grey")

# Reveal true plot
box(col = "red", # Plot a box with red dashed lines
    lty = 2,
    lwd = 2)
which(placement == 20) # Define the position of the box

3) Group comparisons

This plot allows us to visually compare two groups: The dataset provides us information about the vote share for the CDU. It also includes a dummy variable that indicates if the constituency is in Bavaria or not. The following plot compares the vote share for the CDU and distinguishes between constituencies within Bavaria (purple) and outside of Bavaria (green).

We need to generate again the 4x5 grid cells with the random plots and the “true” plot. The following code first plots the 19 random scatter plots. As we can see, we get a 4x5 grid cell with 19 randomly assigned scatter plots and one empty cell. We now proceed and fill again this empty cell with the “true” data and plot a box around it.

# Generate random plot placement
placement <- sample((1:20), 20)
layout(matrix(placement, 4, 5))

# Plot 19 Null Plots
par(mar = c(.1, .1, .1, .1))
for (i in 1:19) {
  random <- sample(c(1:dim(slop)[1]), dim(slop)[1])
  
  plot(
    slop$bayern[random],
    slop$cdu,
    axes = F,
    ann = F,
    cex = .4,
    xlim = c(-1, 2)
    )
  points(1,
         mean(slop$cdu[slop$bayern[random] == 1]),
         pch = "-",
         col = "purple4",
         cex = 3)
  points(0,
         mean(slop$cdu[slop$bayern[random] == 0]),
         pch = "-",
         col = "darkolivegreen2",
         cex = 3)
  box(bty = "l", col = "grey")
  
}

# Add true plot
plot(
  slop$bayern,
  slop$cdu,
  axes = F,
  ann = F,
  cex = .4,
  xlim = c(-1, 2)
  )
points(1,
       mean(slop$cdu[slop$bayern == 1]),
       pch = "-",
       col = "purple4",
       cex = 3)
points(0,
       mean(slop$cdu[slop$bayern == 0]),
       pch = "-",
       col = "darkolivegreen2",
       cex = 3)
box(bty = "l", col = "grey")

# Reveal True Plot
box(col = "red", lty = 2, lwd = 2)
which(placement == 20)