Add P-values and Significance Levels to ggplots

[This article was first published on Easy Guides, and kindly contributed to R-bloggers]. (You can report issue about the content on this page here)
Want to share your content on R-bloggers? click here if you have a blog, or here if you don't.


In this article, we’ll describe how to easily i) compare means of two or multiple groups; ii) and to automatically add p-values and significance levels to a ggplot (such as box plots, dot plots, bar plots and line plots …).

Contents:

Prerequisites

Install and load required R packages

Required R package: ggpubr (version >= 0.1.3), for ggplot2-based publication ready plots.

  • Install from CRAN as follow:
install.packages("ggpubr")
  • Or, install the latest developmental version from GitHub as follow:
if(!require(devtools)) install.packages("devtools")
devtools::install_github("kassambara/ggpubr")
  • Load ggpubr:
library(ggpubr)

Official documentation of ggpubr is available at: http://www.sthda.com/english/rpkgs/ggpubr

Demo data sets

Data: ToothGrowth data sets.

data("ToothGrowth")
head(ToothGrowth)
   len supp dose
1  4.2   VC  0.5
2 11.5   VC  0.5
3  7.3   VC  0.5
4  5.8   VC  0.5
5  6.4   VC  0.5
6 10.0   VC  0.5

Methods for comparing means

The standard methods to compare the means of two or more groups in R, have been largely described at: comparing means in R.

The most common methods for comparing means include:

Methods R function Description
T-test t.test() Compare two groups (parametric)
Wilcoxon test wilcox.test() Compare two groups (non-parametric)
ANOVA aov() or anova() Compare multiple groups (parametric)
Kruskal-Wallis kruskal.test() Compare multiple groups (non-parametric)

A practical guide to compute and interpret the results of each of these methods are provided at the following links:

R functions to add p-values

Here we present two new R functions in the ggpubr package:

  • compare_means(): easy to use solution to performs one and multiple mean comparisons.
  • stat_compare_means(): easy to use solution to automatically add p-values and significance levels to a ggplot.

compare_means()

As we’ll show in the next sections, it has multiple useful options compared to the standard R functions.

The simplified format is as follow:

compare_means(formula, data, method = "wilcox.test", paired = FALSE,
  group.by = NULL, ref.group = NULL, ...)
  • formula: a formula of the form x ~ group, where x is a numeric variable and group is a factor with one or multiple levels. For example, formula = TP53 ~ cancer_group. It’s also possible to perform the test for multiple response variables at the same time. For example, formula = c(TP53, PTEN) ~ cancer_group.
  • data: a data.frame containing the variables in the formula.

  • method: the type of test. Default is “wilcox.test”. Allowed values include:
    • “t.test” (parametric) and “wilcox.test”” (non-parametric). Perform comparison between two groups of samples. If the grouping variable contains more than two levels, then a pairwise comparison is performed.
    • “anova” (parametric) and “kruskal.test” (non-parametric). Perform one-way ANOVA test comparing multiple groups.
  • paired: a logical indicating whether you want a paired test. Used only in t.test and in wilcox.test.

  • group.by: variables used to group the data set before applying the test. When specified the mean comparisons will be performed in each subset of the data formed by the different levels of the group.by variables.

  • ref.group: a character string specifying the reference group. If specified, for a given grouping variable, each of the group levels will be compared to the reference group (i.e. control group). ref.group can be also “.all.”. In this case, each of the grouping variable levels is compared to all (i.e. base-mean).

stat_compare_means()

This function extends ggplot2 for adding mean comparison p-values to a ggplot, such as box blots, dot plots, bar plots and line plots.

The simplified format is as follow:

stat_compare_means(mapping = NULL, comparisons = NULL hide.ns = FALSE,
                   label = NULL,  label.x = NULL, label.y = NULL,  ...)
  • mapping: Set of aesthetic mappings created by aes().

  • comparisons: A list of length-2 vectors. The entries in the vector are either the names of 2 values on the x-axis or the 2 integers that correspond to the index of the groups of interest, to be compared.

  • hide.ns: logical value. If TRUE, hide ns symbol when displaying significance levels.

  • label: character string specifying label type. Allowed values include “p.signif” (shows the significance levels), “p.format” (shows the formatted p value).

  • label.x,label.y: numeric values. coordinates (in data units) to be used for absolute positioning of the label. If too short they will be recycled.

  • : other arguments passed to the function compare_means() such as method, paired, ref.group.

Compare two independent groups

Perform the test:

compare_means(len ~ supp, data = ToothGrowth)
# A tibble: 1 x 8
    .y. group1 group2          p      p.adj p.format p.signif   method
  <chr>  <chr>  <chr>      <dbl>      <dbl>    <chr>    <chr>    <chr>
1   len     OJ     VC 0.06449067 0.06449067    0.064       ns Wilcoxon

By default method = “wilcox.test” (non-parametric test). You can also specify method = “t.test” for a parametric t-test.

Returned value is a data frame with the following columns:

  • .y.: the y variable used in the test.
  • p: the p-value
  • p.adj: the adjusted p-value. Default value for p.adjust.method = “holm”
  • p.format: the formatted p-value
  • p.signif: the significance level.
  • method: the statistical test used to compare groups.

Create a box plot with p-values:

p <- ggboxplot(ToothGrowth, x = "supp", y = "len",
          color = "supp", palette = "jco",
          add = "jitter")
#  Add p-value
p + stat_compare_means()

# Change method
p + stat_compare_means(method = "t.test")
Add p-values and significance levels to ggplotsAdd p-values and significance levels to ggplots

Add p-values and significance levels to ggplots

Note that, the p-value label position can be adjusted using the arguments: label.x, label.y, hjust and vjust.

The default p-value label displayed is obtained by concatenating the method and the p columns of the returned data frame by the function compare_means(). You can specify other combinations using the aes() function.

For example,

  • aes(label = ..p.format..) or aes(label = paste0(“p =”, ..p.format..)): display only the formatted p-value (without the method name)
  • aes(label = ..p.signif..): display only the significance level.
  • aes(label = paste0(..method.., “\n”, “p =”, ..p.format..)): Use line break (“\n”) between the method name and the p-value.

As an illustration, type this:

p + stat_compare_means( aes(label = ..p.signif..), 
                        label.x = 1.5, label.y = 40)
Add p-values and significance levels to ggplots

Add p-values and significance levels to ggplots

If you prefer, it’s also possible to specify the argument label as a character vector:

p + stat_compare_means( label = "p.signif", label.x = 1.5, label.y = 40)

Compare two paired samples

Perform the test:

compare_means(len ~ supp, data = ToothGrowth, paired = TRUE)
# A tibble: 1 x 8
    .y. group1 group2           p       p.adj p.format p.signif   method
  <chr>  <chr>  <chr>       <dbl>       <dbl>    <chr>    <chr>    <chr>
1   len     OJ     VC 0.004312554 0.004312554   0.0043       ** Wilcoxon

Visualize paired data using the ggpaired() function:

ggpaired(ToothGrowth, x = "supp", y = "len",
         color = "supp", line.color = "gray", line.size = 0.4,
         palette = "jco")+
  stat_compare_means(paired = TRUE)
Add p-values and significance levels to ggplots

Add p-values and significance levels to ggplots

Compare more than two groups

  • Global test:
# Global test
compare_means(len ~ dose,  data = ToothGrowth, method = "anova")
# A tibble: 1 x 6
    .y.            p        p.adj p.format p.signif method
  <chr>        <dbl>        <dbl>    <chr>    <chr>  <chr>
1   len 9.532727e-16 9.532727e-16  9.5e-16     ****  Anova

Plot with global p-value:

# Default method = "kruskal.test" for multiple groups
ggboxplot(ToothGrowth, x = "dose", y = "len",
          color = "dose", palette = "jco")+
  stat_compare_means()

# Change method to anova
ggboxplot(ToothGrowth, x = "dose", y = "len",
          color = "dose", palette = "jco")+
  stat_compare_means(method = "anova")
Add p-values and significance levels to ggplotsAdd p-values and significance levels to ggplots

Add p-values and significance levels to ggplots

  • Pairwise comparisons. If the grouping variable contains more than two levels, then pairwise tests will be performed automatically. The default method is “wilcox.test”. You can change this to “t.test”.
# Perorm pairwise comparisons
compare_means(len ~ dose,  data = ToothGrowth)
# A tibble: 3 x 8
    .y. group1 group2            p        p.adj p.format p.signif   method
  <chr>  <chr>  <chr>        <dbl>        <dbl>    <chr>    <chr>    <chr>
1   len    0.5      1 7.020855e-06 1.404171e-05  7.0e-06     **** Wilcoxon
2   len    0.5      2 8.406447e-08 2.521934e-07  8.4e-08     **** Wilcoxon
3   len      1      2 1.772382e-04 1.772382e-04  0.00018      *** Wilcoxon
# Visualize: Specify the comparisons you want
my_comparisons <- list( c("0.5", "1"), c("1", "2"), c("0.5", "2") )
ggboxplot(ToothGrowth, x = "dose", y = "len",
          color = "dose", palette = "jco")+ 
  stat_compare_means(comparisons = my_comparisons)+ # Add pairwise comparisons p-value
  stat_compare_means(label.y = 50)     # Add global p-value
Add p-values and significance levels to ggplots

Add p-values and significance levels to ggplots

If you want to specify the precise y location of bars, use the argument label.y:

ggboxplot(ToothGrowth, x = "dose", y = "len",
          color = "dose", palette = "jco")+ 
  stat_compare_means(comparisons = my_comparisons, label.y = c(29, 35, 40))+
  stat_compare_means(label.y = 45)
Add p-values and significance levels to ggplots

Add p-values and significance levels to ggplots

(Adding bars, connecting compared groups, has been facilitated by the ggsignif R package )

  • Multiple pairwise tests against a reference group:
# Pairwise comparison against reference
compare_means(len ~ dose,  data = ToothGrowth, ref.group = "0.5",
              method = "t.test")
# A tibble: 2 x 8
    .y. group1 group2            p        p.adj p.format p.signif method
  <chr>  <chr>  <chr>        <dbl>        <dbl>    <chr>    <chr>  <chr>
1   len    0.5      1 6.697250e-09 6.697250e-09  6.7e-09     **** T-test
2   len    0.5      2 1.469534e-16 2.939068e-16  < 2e-16     **** T-test
# Visualize
ggboxplot(ToothGrowth, x = "dose", y = "len",
          color = "dose", palette = "jco")+
  stat_compare_means(method = "anova", label.y = 40)+      # Add global p-value
  stat_compare_means(label = "p.signif", method = "t.test",
                     ref.group = "0.5")                    # Pairwise comparison against reference
Add p-values and significance levels to ggplots

Add p-values and significance levels to ggplots

  • Multiple pairwise tests against all (base-mean):
# Comparison of each group against base-mean
compare_means(len ~ dose,  data = ToothGrowth, ref.group = ".all.",
              method = "t.test")
# A tibble: 3 x 8
    .y. group1 group2            p        p.adj p.format p.signif method
  <chr>  <chr>  <chr>        <dbl>        <dbl>    <chr>    <chr>  <chr>
1   len  .all.    0.5 1.244626e-06 3.733877e-06  1.2e-06     **** T-test
2   len  .all.      1 5.667266e-01 5.667266e-01     0.57       ns T-test
3   len  .all.      2 1.371704e-05 2.743408e-05  1.4e-05     **** T-test
# Visualize
ggboxplot(ToothGrowth, x = "dose", y = "len",
          color = "dose", palette = "jco")+
  stat_compare_means(method = "anova", label.y = 40)+      # Add global p-value
  stat_compare_means(label = "p.signif", method = "t.test",
                     ref.group = ".all.")                  # Pairwise comparison against all
Add p-values and significance levels to ggplots

Add p-values and significance levels to ggplots

A typical situation, where pairwise comparisons against “all” can be useful, is illustrated here using the myeloma data set from the survminer package.

We’ll plot the expression profile of the DEPDC1 gene according to the patients’ molecular groups. We want to know if there is any difference between groups. If yes, where the difference is?

To answer to this question, you can perform a pairwise comparison between all the 7 groups. This will lead to a lot of comparisons between all possible combinations. If you have many groups, as here, it might be difficult to interpret.

Another easy solution is to compare each of the seven groups against “all” (i.e. base-mean). When the test is significant, then you can conclude that DEPDC1 is significantly overexpressed or downexpressed in a group xxx compared to all.

# Load myeloma data from survminer package
if(!require(survminer)) install.packages("survminer")
data("myeloma", package = "survminer")

# Perform the test
compare_means(DEPDC1 ~ molecular_group,  data = myeloma,
              ref.group = ".all.", method = "t.test")
# A tibble: 7 x 8
     .y. group1           group2            p        p.adj p.format p.signif method
   <chr>  <chr>            <chr>        <dbl>        <dbl>    <chr>    <chr>  <chr>
1 DEPDC1  .all.       Cyclin D-1 1.496896e-01 4.490687e-01  0.14969       ns T-test
2 DEPDC1  .all.       Cyclin D-2 5.231428e-01 1.000000e+00  0.52314       ns T-test
3 DEPDC1  .all.     Hyperdiploid 2.815333e-04 1.689200e-03  0.00028      *** T-test
4 DEPDC1  .all. Low bone disease 5.083985e-03 2.541992e-02  0.00508       ** T-test
5 DEPDC1  .all.              MAF 8.610664e-02 3.444265e-01  0.08611       ns T-test
6 DEPDC1  .all.            MMSET 5.762908e-01 1.000000e+00  0.57629       ns T-test
7 DEPDC1  .all.    Proliferation 1.241416e-09 8.689910e-09  1.2e-09     **** T-test
# Visualize the expression profile
ggboxplot(myeloma, x = "molecular_group", y = "DEPDC1", color = "molecular_group", 
          add = "jitter", legend = "none") +
  rotate_x_text(angle = 45)+
  geom_hline(yintercept = mean(myeloma$DEPDC1), linetype = 2)+ # Add horizontal line at base mean
  stat_compare_means(method = "anova", label.y = 1600)+        # Add global annova p-value
  stat_compare_means(label = "p.signif", method = "t.test",
                     ref.group = ".all.")                      # Pairwise comparison against all
Add p-values and significance levels to ggplots

Add p-values and significance levels to ggplots

From the plot above, we can conclude that DEPDC1 is significantly overexpressed in proliferation group and, it’s significantly downexpressed in Hyperdiploid and Low bone disease compared to all.

Note that, if you want to hide the ns symbol, specify the argument hide.ns = TRUE.

# Visualize the expression profile
ggboxplot(myeloma, x = "molecular_group", y = "DEPDC1", color = "molecular_group", 
          add = "jitter", legend = "none") +
  rotate_x_text(angle = 45)+
  geom_hline(yintercept = mean(myeloma$DEPDC1), linetype = 2)+ # Add horizontal line at base mean
  stat_compare_means(method = "anova", label.y = 1600)+        # Add global annova p-value
  stat_compare_means(label = "p.signif", method = "t.test",
                     ref.group = ".all.", hide.ns = TRUE)      # Pairwise comparison against all
Add p-values and significance levels to ggplots

Add p-values and significance levels to ggplots

Multiple grouping variables

  • Two independent sample comparisons after grouping the data by another variable:

Perform the test:

compare_means(len ~ supp, data = ToothGrowth, 
              group.by = "dose")
# A tibble: 3 x 9
   dose   .y. group1 group2           p      p.adj p.format p.signif   method
  <dbl> <chr>  <chr>  <chr>       <dbl>      <dbl>    <chr>    <chr>    <chr>
1   0.5   len     OJ     VC 0.023186427 0.04637285    0.023        * Wilcoxon
2   1.0   len     OJ     VC 0.004030367 0.01209110    0.004       ** Wilcoxon
3   2.0   len     OJ     VC 1.000000000 1.00000000    1.000       ns Wilcoxon

In the example above, for each level of the variable “dose”, we compare the means of the variable “len” in the different groups formed by the grouping variable “supp”.

Visualize (1/2). Create a multi-panel box plots facetted by group (here, “dose”):

# Box plot facetted by "dose"
p <- ggboxplot(ToothGrowth, x = "supp", y = "len",
          color = "supp", palette = "jco",
          add = "jitter",
          facet.by = "dose", short.panel.labs = FALSE)
# Use only p.format as label. Remove method name.
p + stat_compare_means(label = "p.format")
Add p-values and significance levels to ggplots

Add p-values and significance levels to ggplots

# Or use significance symbol as label
p + stat_compare_means(label =  "p.signif", label.x = 1.5)
Add p-values and significance levels to ggplots

Add p-values and significance levels to ggplots

To hide the ‘ns’ symbol, use the argument hide.ns = TRUE.

Visualize (2/2). Create one single panel with all box plots. Plot y = “len” by x = “dose” and color by “supp”:

p <- ggboxplot(ToothGrowth, x = "dose", y = "len",
          color = "supp", palette = "jco",
          add = "jitter")
p + stat_compare_means(aes(group = supp))
Add p-values and significance levels to ggplots

Add p-values and significance levels to ggplots

# Show only p-value
p + stat_compare_means(aes(group = supp), label = "p.format")
Add p-values and significance levels to ggplots

Add p-values and significance levels to ggplots

# Use significance symbol as label
p + stat_compare_means(aes(group = supp), label = "p.signif")
Add p-values and significance levels to ggplots

Add p-values and significance levels to ggplots

  • Paired sample comparisons after grouping the data by another variable:

Perform the test:

compare_means(len ~ supp, data = ToothGrowth, 
              group.by = "dose", paired = TRUE)
# A tibble: 3 x 9
   dose   .y. group1 group2          p      p.adj p.format p.signif   method
  <dbl> <chr>  <chr>  <chr>      <dbl>      <dbl>    <chr>    <chr>    <chr>
1   0.5   len     OJ     VC 0.03296938 0.06593876    0.033        * Wilcoxon
2   1.0   len     OJ     VC 0.01905889 0.05717667    0.019        * Wilcoxon
3   2.0   len     OJ     VC 1.00000000 1.00000000    1.000       ns Wilcoxon

Visualize. Create a multi-panel box plots facetted by group (here, “dose”):

# Box plot facetted by "dose"
p <- ggpaired(ToothGrowth, x = "supp", y = "len",
          color = "supp", palette = "jco", 
          line.color = "gray", line.size = 0.4,
          facet.by = "dose", short.panel.labs = FALSE)
# Use only p.format as label. Remove method name.
p + stat_compare_means(label = "p.format", paired = TRUE)
Add p-values and significance levels to ggplots

Add p-values and significance levels to ggplots

Other plot types

  • Bar and line plots (one grouping variable):
# Bar plot of mean +/-se
ggbarplot(ToothGrowth, x = "dose", y = "len", add = "mean_se")+
  stat_compare_means() +                                         # Global p-value
  stat_compare_means(ref.group = "0.5", label = "p.signif",
                     label.y = c(22, 29))                   # compare to ref.group

# Line plot of mean +/-se
ggline(ToothGrowth, x = "dose", y = "len", add = "mean_se")+
  stat_compare_means() +                                         # Global p-value
  stat_compare_means(ref.group = "0.5", label = "p.signif",
                     label.y = c(22, 29))     
Add p-values and significance levels to ggplotsAdd p-values and significance levels to ggplots

Add p-values and significance levels to ggplots

  • Bar and line plots (two grouping variables):
ggbarplot(ToothGrowth, x = "dose", y = "len", add = "mean_se",
          color = "supp", palette = "jco", 
          position = position_dodge(0.8))+
  stat_compare_means(aes(group = supp), label = "p.signif", label.y = 29)

ggline(ToothGrowth, x = "dose", y = "len", add = "mean_se",
          color = "supp", palette = "jco")+
  stat_compare_means(aes(group = supp), label = "p.signif", 
                     label.y = c(16, 25, 29))
Add p-values and significance levels to ggplotsAdd p-values and significance levels to ggplots

Add p-values and significance levels to ggplots

Infos

This analysis has been performed using R software (ver. 3.3.2) and ggpubr (ver. 0.1.3).

To leave a comment for the author, please follow the link and comment on their blog: Easy Guides.

R-bloggers.com offers daily e-mail updates about R news and tutorials about learning R and many other topics. Click here if you're looking to post or find an R/data-science job.
Want to share your content on R-bloggers? click here if you have a blog, or here if you don't.

Never miss an update!
Subscribe to R-bloggers to receive
e-mails with the latest R posts.
(You will not see this message again.)

Click here to close (This popup will not appear again)