Variable vs. Participant-wise Standardization

[This article was first published on Dominique Makowski, and kindly contributed to R-bloggers]. (You can report issue about the content on this page here)
Want to share your content on R-bloggers? click here if you have a blog, or here if you don't.

To make sense of their data and effects, psychologists often standardize (Z-score) their variables. However, in repeated-measures designs, there are three ways of standardizing data:

  • Variable-wise: The most common method. A simple scaling and reducing of each variable by their mean and SD.
  • Participant-wise: Variables are standardized “within” each participant, i.e., for each participant, by the participant’s mean and SD.
  • Full: Participant-wise first and then re-standardizing variable-wise.

Unfortunately, the method used is often not explicitly stated. This is an issue as these methods can generate important discrepancies that contribute to the reproducibility crisis of psychological science.

In the following, we will see how to perform those methods and look for differences.

The data

We will take a dataset in which participants were exposed to negative pictures and had to rate their emotions (valence) and the amount of memories associated with the picture (autobiographical link). One could make the hypothesis that for young participants with no context of war or violence, the most negative pictures (mutilations) are less related to memories than less negative pictures (involving for example car crashes or sick people). In other words, we expect a positive relationship between valence (with high values corresponding to less negativity) and autobiographical link.

Let’s have a look at the data, averaged by participants:

<span class="n">library</span><span class="p">(</span><span class="n">tidyverse</span><span class="p">)</span><span class="w">
</span><span class="n">library</span><span class="p">(</span><span class="n">psycho</span><span class="p">)</span><span class="w">

</span><span class="n">df</span><span class="w"> </span><span class="o"><-</span><span class="w"> </span><span class="n">psycho</span><span class="o">::</span><span class="n">emotion</span><span class="w"> </span><span class="o">%>%</span><span class="w">  </span><span class="c1"># Load the dataset from the psycho package</span><span class="w">
  </span><span class="n">filter</span><span class="p">(</span><span class="n">Emotion_Condition</span><span class="w"> </span><span class="o">==</span><span class="w"> </span><span class="s2">"Negative"</span><span class="p">)</span><span class="w">  </span><span class="c1"># Discard neutral pictures</span><span class="w">

</span><span class="n">df</span><span class="w"> </span><span class="o">%>%</span><span class="w"> 
  </span><span class="n">group_by</span><span class="p">(</span><span class="n">Participant_ID</span><span class="p">)</span><span class="w"> </span><span class="o">%>%</span><span class="w"> 
  </span><span class="n">summarise</span><span class="p">(</span><span class="n">n_Trials</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">n</span><span class="p">(),</span><span class="w">
            </span><span class="n">Valence_Mean</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">mean</span><span class="p">(</span><span class="n">Subjective_Valence</span><span class="p">,</span><span class="w"> </span><span class="n">na.rm</span><span class="o">=</span><span class="kc">TRUE</span><span class="p">),</span><span class="w">
            </span><span class="n">Valence_SD</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">sd</span><span class="p">(</span><span class="n">Subjective_Valence</span><span class="p">,</span><span class="w"> </span><span class="n">na.rm</span><span class="o">=</span><span class="kc">TRUE</span><span class="p">),</span><span class="w">
            </span><span class="n">Autobiographical_Link_Mean</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">mean</span><span class="p">(</span><span class="n">Autobiographical_Link</span><span class="p">,</span><span class="w"> </span><span class="n">na.rm</span><span class="o">=</span><span class="kc">TRUE</span><span class="p">),</span><span class="w">
            </span><span class="n">Autobiographical_Link_SD</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">sd</span><span class="p">(</span><span class="n">Autobiographical_Link</span><span class="p">,</span><span class="w"> </span><span class="n">na.rm</span><span class="o">=</span><span class="kc">TRUE</span><span class="p">))</span><span class="w"> </span><span class="o">%>%</span><span class="w"> 
  </span><span class="n">mutate</span><span class="p">(</span><span class="n">Valence</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">paste</span><span class="p">(</span><span class="n">Valence_Mean</span><span class="p">,</span><span class="w"> </span><span class="n">Valence_SD</span><span class="p">,</span><span class="w"> </span><span class="n">sep</span><span class="o">=</span><span class="s2">" +- "</span><span class="p">),</span><span class="w">
         </span><span class="n">Autobiographical_Link</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">paste</span><span class="p">(</span><span class="n">Autobiographical_Link_Mean</span><span class="p">,</span><span class="w"> </span><span class="n">Autobiographical_Link_SD</span><span class="p">,</span><span class="w"> </span><span class="n">sep</span><span class="o">=</span><span class="s2">" +- "</span><span class="p">))</span><span class="w"> </span><span class="o">%>%</span><span class="w"> 
  </span><span class="n">select</span><span class="p">(</span><span class="o">-</span><span class="n">ends_with</span><span class="p">(</span><span class="s2">"SD"</span><span class="p">),</span><span class="w"> </span><span class="o">-</span><span class="n">ends_with</span><span class="p">(</span><span class="s2">"Mean"</span><span class="p">))</span><span class="w">
</span>
Participant_ID n_Trials Valence Autobiographical_Link
10S 24 -58.07 +- 42.59 49.88 +- 29.60
11S 24 -73.22 +- 37.01 0.94 +- 3.46
12S 24 -57.53 +- 26.56 30.24 +- 27.87
13S 24 -63.22 +- 23.72 27.86 +- 35.81
14S 24 -56.60 +- 26.47 3.31 +- 11.20
15S 24 -60.59 +- 33.71 17.64 +- 17.36
16S 24 -46.12 +- 24.88 15.33 +- 16.57
17S 24 -1.54 +- 4.98 0.13 +- 0.64
18S 24 -67.23 +- 34.98 22.71 +- 20.07
19S 24 -59.61 +- 33.22 28.37 +- 12.55

As we can see from the means and SDs, there is a lot of variability between and within participants.

Standardize

We will create three dataframes standardized with each of the three techniques.

<span class="n">Z_VarWise</span><span class="w"> </span><span class="o"><-</span><span class="w"> </span><span class="n">df</span><span class="w"> </span><span class="o">%>%</span><span class="w"> 
  </span><span class="n">standardize</span><span class="p">()</span><span class="w">

</span><span class="n">Z_ParWise</span><span class="w"> </span><span class="o"><-</span><span class="w"> </span><span class="n">df</span><span class="w"> </span><span class="o">%>%</span><span class="w"> 
  </span><span class="n">group_by</span><span class="p">(</span><span class="n">Participant_ID</span><span class="p">)</span><span class="w"> </span><span class="o">%>%</span><span class="w"> 
  </span><span class="n">standardize</span><span class="p">()</span><span class="w"> 

</span><span class="n">Z_Full</span><span class="w"> </span><span class="o"><-</span><span class="w"> </span><span class="n">df</span><span class="w"> </span><span class="o">%>%</span><span class="w"> 
  </span><span class="n">group_by</span><span class="p">(</span><span class="n">Participant_ID</span><span class="p">)</span><span class="w"> </span><span class="o">%>%</span><span class="w"> 
  </span><span class="n">standardize</span><span class="p">()</span><span class="w"> </span><span class="o">%>%</span><span class="w"> 
  </span><span class="n">ungroup</span><span class="p">()</span><span class="w"> </span><span class="o">%>%</span><span class="w"> 
  </span><span class="n">standardize</span><span class="p">()</span><span class="w"> 
</span>

Effect of Standardization

Let’s see how these three standardization techniques affected the Valence variable.

At a general level

<span class="c1"># Create convenient function</span><span class="w">
</span><span class="n">print_summary</span><span class="w"> </span><span class="o"><-</span><span class="w"> </span><span class="k">function</span><span class="p">(</span><span class="n">data</span><span class="p">){</span><span class="w">
  </span><span class="n">paste</span><span class="p">(</span><span class="n">deparse</span><span class="p">(</span><span class="nf">substitute</span><span class="p">(</span><span class="n">data</span><span class="p">)),</span><span class="w"> </span><span class="s2">":"</span><span class="p">,</span><span class="w"> 
        </span><span class="n">format_digit</span><span class="p">(</span><span class="n">mean</span><span class="p">(</span><span class="n">data</span><span class="p">[[</span><span class="s2">"Subjective_Valence"</span><span class="p">]])),</span><span class="w">
        </span><span class="s2">"+-"</span><span class="p">,</span><span class="w">
        </span><span class="n">format_digit</span><span class="p">(</span><span class="n">sd</span><span class="p">(</span><span class="n">data</span><span class="p">[[</span><span class="s2">"Subjective_Valence"</span><span class="p">]])),</span><span class="w">
        </span><span class="s2">"["</span><span class="p">,</span><span class="w"> </span><span class="n">format_digit</span><span class="p">(</span><span class="nf">min</span><span class="p">(</span><span class="n">data</span><span class="p">[[</span><span class="s2">"Subjective_Valence"</span><span class="p">]])),</span><span class="w">
        </span><span class="s2">","</span><span class="p">,</span><span class="w"> </span><span class="n">format_digit</span><span class="p">(</span><span class="nf">max</span><span class="p">(</span><span class="n">data</span><span class="p">[[</span><span class="s2">"Subjective_Valence"</span><span class="p">]])),</span><span class="w">
        </span><span class="s2">"]"</span><span class="p">)</span><span class="w">
</span><span class="p">}</span><span class="w">

</span><span class="c1"># Check the results</span><span class="w">
</span><span class="n">print_summary</span><span class="p">(</span><span class="n">Z_VarWise</span><span class="p">)</span><span class="w">
</span>
[1] "Z_VarWise : 0 +- 1.00 [ -1.18 , 3.10 ]"
<span class="n">print_summary</span><span class="p">(</span><span class="n">Z_ParWise</span><span class="p">)</span><span class="w">
</span>
[1] "Z_ParWise : 0 +- 0.98 [ -2.93 , 3.29 ]"
<span class="n">print_summary</span><span class="p">(</span><span class="n">Z_Full</span><span class="p">)</span><span class="w">
</span>
[1] "Z_Full : 0 +- 1.00 [ -2.99 , 3.36 ]"

At a participant level

<span class="c1"># Create convenient function</span><span class="w">
</span><span class="n">print_participants</span><span class="w"> </span><span class="o"><-</span><span class="w"> </span><span class="k">function</span><span class="p">(</span><span class="n">data</span><span class="p">){</span><span class="w">
  </span><span class="n">data</span><span class="w"> </span><span class="o">%>%</span><span class="w"> 
    </span><span class="n">group_by</span><span class="p">(</span><span class="n">Participant_ID</span><span class="p">)</span><span class="w"> </span><span class="o">%>%</span><span class="w"> 
    </span><span class="n">summarise</span><span class="p">(</span><span class="n">Mean</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">mean</span><span class="p">(</span><span class="n">Subjective_Valence</span><span class="p">),</span><span class="w"> 
              </span><span class="n">SD</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">sd</span><span class="p">(</span><span class="n">Subjective_Valence</span><span class="p">))</span><span class="w"> </span><span class="o">%>%</span><span class="w"> 
    </span><span class="n">mutate_if</span><span class="p">(</span><span class="n">is.numeric</span><span class="p">,</span><span class="w"> </span><span class="n">round</span><span class="p">,</span><span class="w"> </span><span class="m">2</span><span class="p">)</span><span class="w"> </span><span class="o">%>%</span><span class="w"> 
    </span><span class="n">head</span><span class="p">(</span><span class="m">5</span><span class="p">)</span><span class="w"> 
    
</span><span class="p">}</span><span class="w">

</span><span class="c1"># Check the results</span><span class="w">
</span><span class="n">print_participants</span><span class="p">(</span><span class="n">Z_VarWise</span><span class="p">)</span><span class="w">
</span>
# A tibble: 5 x 3
  Participant_ID    Mean    SD
  <fct>            <dbl> <dbl>
1 10S            -0.0500 1.15 
2 11S            -0.460  1.00 
3 12S            -0.0300 0.720
4 13S            -0.190  0.640
5 14S            -0.0100 0.710
<span class="n">print_participants</span><span class="p">(</span><span class="n">Z_ParWise</span><span class="p">)</span><span class="w">
</span>
# A tibble: 5 x 3
  Participant_ID  Mean    SD
  <fct>          <dbl> <dbl>
1 10S               0.    1.
2 11S               0.    1.
3 12S               0.    1.
4 13S               0.    1.
5 14S               0.    1.
<span class="n">print_participants</span><span class="p">(</span><span class="n">Z_Full</span><span class="p">)</span><span class="w">
</span>
# A tibble: 5 x 3
  Participant_ID  Mean    SD
  <fct>          <dbl> <dbl>
1 10S               0.  1.02
2 11S               0.  1.02
3 12S               0.  1.02
4 13S               0.  1.02
5 14S               0.  1.02

Distribution

<span class="n">data.frame</span><span class="p">(</span><span class="n">VarWise</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">Z_VarWise</span><span class="o">$</span><span class="n">Subjective_Valence</span><span class="p">,</span><span class="w">
           </span><span class="n">ParWise</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">Z_ParWise</span><span class="o">$</span><span class="n">Subjective_Valence</span><span class="p">,</span><span class="w">
           </span><span class="n">Full</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">Z_Full</span><span class="o">$</span><span class="n">Subjective_Valence</span><span class="p">)</span><span class="w"> </span><span class="o">%>%</span><span class="w"> 
  </span><span class="n">gather</span><span class="p">(</span><span class="n">Method</span><span class="p">,</span><span class="w"> </span><span class="n">Variable</span><span class="p">)</span><span class="w"> </span><span class="o">%>%</span><span class="w"> 
  </span><span class="n">ggplot</span><span class="p">(</span><span class="n">aes</span><span class="p">(</span><span class="n">x</span><span class="o">=</span><span class="n">Variable</span><span class="p">,</span><span class="w"> </span><span class="n">fill</span><span class="o">=</span><span class="n">Method</span><span class="p">))</span><span class="w"> </span><span class="o">+</span><span class="w">
  </span><span class="n">geom_density</span><span class="p">(</span><span class="n">alpha</span><span class="o">=</span><span class="m">0.5</span><span class="p">)</span><span class="w"> </span><span class="o">+</span><span class="w">
  </span><span class="n">theme_minimal</span><span class="p">()</span><span class="w"> 
</span>

The distributions appear to be similar…

Correlation

Let’s do a correlation between the variable-wise and participant-wise methods.

<span class="n">psycho</span><span class="o">::</span><span class="n">bayes_cor.test</span><span class="p">(</span><span class="n">Z_VarWise</span><span class="o">$</span><span class="n">Subjective_Valence</span><span class="p">,</span><span class="w"> </span><span class="n">Z_ParWise</span><span class="o">$</span><span class="n">Subjective_Valence</span><span class="p">)</span><span class="w">
</span>
Results of the Bayesian correlation indicate moderate evidence (BF = 8.65) in favour of an absence of a negative association between Z_VarWise$Subjective_Valence and Z_ParWise$Subjective_Valence (r = -0.015, MAD = 0.047, 90% CI [-0.096, 0.057]). The correlation can be considered as small or very small with respective probabilities of 3.46% and 59.43%.
<span class="n">data.frame</span><span class="p">(</span><span class="n">Original</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">df</span><span class="o">$</span><span class="n">Subjective_Valence</span><span class="p">,</span><span class="w">
           </span><span class="n">VarWise</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">Z_VarWise</span><span class="o">$</span><span class="n">Subjective_Valence</span><span class="p">,</span><span class="w">
           </span><span class="n">ParWise</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">Z_ParWise</span><span class="o">$</span><span class="n">Subjective_Valence</span><span class="p">)</span><span class="w"> </span><span class="o">%>%</span><span class="w"> 
  </span><span class="n">ggplot</span><span class="p">(</span><span class="n">aes</span><span class="p">(</span><span class="n">x</span><span class="o">=</span><span class="n">VarWise</span><span class="p">,</span><span class="w"> </span><span class="n">y</span><span class="o">=</span><span class="n">ParWise</span><span class="p">,</span><span class="w"> </span><span class="n">colour</span><span class="o">=</span><span class="n">Original</span><span class="p">))</span><span class="w"> </span><span class="o">+</span><span class="w">
  </span><span class="n">geom_point</span><span class="p">()</span><span class="w"> </span><span class="o">+</span><span class="w">
  </span><span class="n">geom_smooth</span><span class="p">(</span><span class="n">method</span><span class="o">=</span><span class="s2">"lm"</span><span class="p">)</span><span class="w"> </span><span class="o">+</span><span class="w">
  </span><span class="n">theme_minimal</span><span class="p">()</span><span class="w">
</span>

While the three standardization methods roughly present the same characteristics at a general level (mean 0 and SD 1) and a similar distribution, their values are very different and completely uncorrelated!

Test

Let’s now answer to the original question by investigating the linear relationship between valence and autobiographical link. We can do this by running a mixed model with participants entered as random effects.

<span class="c1"># Convenient function</span><span class="w">
</span><span class="n">print_model</span><span class="w"> </span><span class="o"><-</span><span class="w"> </span><span class="k">function</span><span class="p">(</span><span class="n">data</span><span class="p">){</span><span class="w">
  </span><span class="n">type_name</span><span class="w"> </span><span class="o"><-</span><span class="w"> </span><span class="n">deparse</span><span class="p">(</span><span class="nf">substitute</span><span class="p">(</span><span class="n">data</span><span class="p">))</span><span class="w"> 

  </span><span class="n">lmerTest</span><span class="o">::</span><span class="n">lmer</span><span class="p">(</span><span class="n">Subjective_Valence</span><span class="w"> </span><span class="o">~</span><span class="w"> </span><span class="n">Autobiographical_Link</span><span class="w"> </span><span class="o">+</span><span class="w"> </span><span class="p">(</span><span class="m">1</span><span class="o">|</span><span class="n">Participant_ID</span><span class="p">),</span><span class="w"> </span><span class="n">data</span><span class="o">=</span><span class="n">data</span><span class="p">)</span><span class="w"> </span><span class="o">%>%</span><span class="w"> 
    </span><span class="n">psycho</span><span class="o">::</span><span class="n">analyze</span><span class="p">(</span><span class="n">CI</span><span class="o">=</span><span class="kc">NULL</span><span class="p">)</span><span class="w"> </span><span class="o">%>%</span><span class="w">
    </span><span class="n">summary</span><span class="p">()</span><span class="w"> </span><span class="o">%>%</span><span class="w"> 
    </span><span class="n">filter</span><span class="p">(</span><span class="n">Variable</span><span class="w"> </span><span class="o">==</span><span class="w"> </span><span class="s2">"Autobiographical_Link"</span><span class="p">)</span><span class="w"> </span><span class="o">%>%</span><span class="w"> 
    </span><span class="n">mutate</span><span class="p">(</span><span class="n">Type</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">type_name</span><span class="p">,</span><span class="w">
           </span><span class="n">Coef</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nf">round</span><span class="p">(</span><span class="n">Coef</span><span class="p">,</span><span class="w"> </span><span class="m">2</span><span class="p">),</span><span class="w">
           </span><span class="n">p</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">format_p</span><span class="p">(</span><span class="n">p</span><span class="p">))</span><span class="w"> </span><span class="o">%>%</span><span class="w"> 
    </span><span class="n">select</span><span class="p">(</span><span class="n">Type</span><span class="p">,</span><span class="w"> </span><span class="n">Coef</span><span class="p">,</span><span class="w"> </span><span class="n">p</span><span class="p">)</span><span class="w">
</span><span class="p">}</span><span class="w">

</span><span class="c1"># Run the model on all datasets</span><span class="w">
</span><span class="n">rbind</span><span class="p">(</span><span class="n">print_model</span><span class="p">(</span><span class="n">df</span><span class="p">),</span><span class="w"> 
      </span><span class="n">print_model</span><span class="p">(</span><span class="n">Z_VarWise</span><span class="p">),</span><span class="w">
      </span><span class="n">print_model</span><span class="p">(</span><span class="n">Z_ParWise</span><span class="p">),</span><span class="w">
      </span><span class="n">print_model</span><span class="p">(</span><span class="n">Z_Full</span><span class="p">))</span><span class="w">
</span>
       Type Coef       p
1        df 0.09    > .1
2 Z_VarWise 0.07    > .1
3 Z_ParWise 0.08 = 0.08°
4    Z_Full 0.08 = 0.08°

As we can see, in our case, using participant-wise standardization resulted in a significant (at p = .1) effect! But keep in mind that this is not always the case. In can be the contrary, or generate very similar results. No method is better or more justified, and its choice depends on the specific case, context, data and goal.

Conclusion

  1. Standardization can be useful in some cases and should be justified
  2. Variable and Participant-wise standardization methods produce “in appearance” similar data
  3. Variable and Participant-wise standardization can lead to different and uncorrelated results
  4. The choice of the method can strongly influence the results and thus, should be explicitly stated

We showed here yet another way of sneakily tweaking the data that can change the results. To prevent its use for p-hacking, we can only support the generalization of open-data, open-analysis and preregistration.

Credits

The psycho package helped you? Don’t forget to cite the various packages you used 🙂

You can cite psycho as follows:

  • Makowski, (2018). The psycho Package: An Efficient and Publishing-Oriented Workflow for Psychological Science. Journal of Open Source Software, 3(22), 470. https://doi.org/10.21105/joss.00470

Previous blogposts

To leave a comment for the author, please follow the link and comment on their blog: Dominique Makowski.

R-bloggers.com offers daily e-mail updates about R news and tutorials about learning R and many other topics. Click here if you're looking to post or find an R/data-science job.
Want to share your content on R-bloggers? click here if you have a blog, or here if you don't.

Never miss an update!
Subscribe to R-bloggers to receive
e-mails with the latest R posts.
(You will not see this message again.)

Click here to close (This popup will not appear again)