% cochran_qtest(opinion ~ time | respondent) ## # A tibble: 1 × 6 ## .y. n statistic df p method ## * ## 1 opinion 200 18.3 2 0.000108 Cochran's Q test The output shows: the variable of interest (.y.), the number of subjects (n), the value of the test statistic \(Q\) (statistic), the degrees of freedom (df), equal to \(k - 1 = 2\) in our case since we compare 3 measurements, the \(p\)-value (p) and the name of the test which has been performed (method). Note that the CochranQTest() function from the {DescTools} package is an alternative to perform this test. As mentioned in the previous section, the Cochran’s Q test reduces to the McNemar’s test when only two related measurements are compared. This is easily verified on our data by keeping only the first two points in time: # Cochran's Q test on the first 2 measurements only dat_long %>% filter(time %in% c("before", "after")) %>% mutate(time = droplevels(time)) %>% cochran_qtest(opinion ~ time | respondent) ## # A tibble: 1 × 6 ## .y. n statistic df p method ## * ## 1 opinion 200 17.3 1 0.0000318 Cochran's Q test # McNemar's test on the same 2 measurements mcnemar.test(table(dat$before, dat$after), correct = FALSE ) ## ## McNemar's Chi-squared test ## ## data: table(dat$before, dat$after) ## McNemar's chi-squared = 17.308, df = 1, p-value = 3.179e-05 The two test statistics (and the two \(p\)-values) are identical.1 It is the \(p\)-value which is of interest to conclude the test. If you are not familiar with \(p\)-values, I invite you to read this section. Interpretations Based on the Cochran’s Q test, we reject the null hypothesis at the significance level \(\alpha = 0.05\) and we conclude that the proportion of citizens in favor of the new policy is not the same at the three points in time (\(p\)-value < 0.001). (For the sake of illustration, if the \(p\)-value had been larger than the significance level \(\alpha = 0.05\): we could not have rejected the null hypothesis, so we could not have concluded that the proportion of citizens in favor of the policy changed over time.) Note also that the test does not indicate the direction of the change, which must be read from the proportions and the plot presented in the section about the data. Post-hoc tests We have just showed that the proportion of citizens in favor of the policy is not stable over time. Nonetheless, here comes the limitation of the test: it does not say which measurement(s) differ(s) from the others. To know this, we need post-hoc tests (in Latin, “after this”, so after obtaining significant results for the Cochran’s Q test), also referred as multiple pairwise-comparison tests. The logic is the same as the one presented for the Kruskal-Wallis test: we compare the groups two by two, and we adjust the \(p\)-values because performing several tests on the same data increases the risk of finding a significant difference by chance alone. Here, the natural post-hoc test is simply the McNemar’s test applied to each pair of measurements. The post-hoc step is thus literally a repeated application of the test presented in the article dedicated to the McNemar’s test, which makes sense given that the Cochran’s Q test is nothing more than its extension to more than two related measurements. With 3 measurements, there are 3 pairs to compare. This is done with the pairwise_mcnemar_test() function of the {rstatix} package, with the Holm method to adjust the \(p\)-values:2 dat_long %>% pairwise_mcnemar_test(opinion ~ time | respondent, p.adjust.method = "holm" ) ## # A tibble: 3 × 8 ## group1 group2 statistic df p p.adj p.adj.signif method ## * ## 1 before after 16.2 1 0.0000578 0.000173 *** McNemar test ## 2 before follow_up 3.31 1 0.0689 0.0689 ns McNemar test ## 3 after follow_up 6.04 1 0.0140 0.0280 * McNemar test It is the p.adj column (the \(p\)-values adjusted for multiple comparisons) which is of interest, and not the p column (the unadjusted \(p\)-values). These adjusted \(p\)-values must be compared to the desired significance level (usually 5%). Based on the output, we conclude that: the proportion of citizens in favor of the policy differs significantly between before and right after the debate (\(p\)-value < 0.001), it differs significantly between right after the debate and one month later (\(p\)-value = 0.028), and it does not differ significantly between before the debate and one month later (\(p\)-value = 0.069). Combined with the proportions computed earlier, these post-hoc tests give a much more precise picture than the Cochran’s Q test alone: the debate significantly increased the support for the policy in the short run (from 46% to 61%), but this increase did not last. One month later, the support had significantly decreased compared to the level observed right after the debate, and it was back to a level no longer significantly different from the initial one. Summary In this article, we reviewed the aim, the hypotheses and the assumptions of the Cochran’s Q test, used to compare three or more related proportions. We then showed how to perform it in R with the cochran_qtest() function of the {rstatix} package, how to interpret its results by comparing the \(p\)-value with the significance level \(\alpha\), and, since a significant result only indicates that at least one measurement differs from the others, how to identify which ones thanks to pairwise McNemar’s tests with adjusted \(p\)-values. Remember, last but not least, that the McNemar’s test is the special case of the Cochran’s Q test for exactly two related measurements, and that with independent samples the Chi-square test of independence should be preferred. Thanks for reading. I hope this article helped you to understand the Cochran’s Q test and how to perform it in R. As always, if you have a question or a suggestion related to the topic covered in this article, please add it as a comment so other readers can benefit from the discussion. The continuity correction must be removed with correct = FALSE for the equality to hold, since the Cochran’s Q test does not apply such a correction.↩︎ The Holm adjustment is less conservative than the Bonferroni one, which is the default in this function. See ?p.adjust for the other available methods.↩︎ " />

socviz 2.0.0 on CRAN

August 21, 2026 | R on kieranhealy.org

In anticipation of the second edition of Data Visualization, which is coming later this year from Princeton University Press, version 2.0.0 of my socviz package is now on CRAN. The update removes some functions that aren’t needed anymore and adds... [Read more...]

McNemar’s test in R

August 18, 2026 | R on Stats and R

Introduction In a previous article, we showed how to perform the Chi-square test of independence in R in order to test whether two qualitative variables are related. As mentioned in that article (and in the one showing how to do the Chi-square tes...
[Read more...]

A Summer, Explained with R

August 18, 2026 | The Jumping Rivers Blog

It’s 2:47pm, the meeting has been running for 38 minutes, and someone has just said, “Can everyone see my screen?” You’re trying to concentrate, your laptop fan sounds like it’s preparing for take-off, and somewhere outside the temperature has climbed to 34°C. You’ve already relocated once today ...
[Read more...]

Breaking the Python Barrier: Building a Pure R-Native DeepAR Engine with LibTorch

August 18, 2026 | Selcuk Disci

Deep learning for time series forecasting in R has historically faced a major architectural hurdle: Python overhead. Frameworks like modeltime.gluonts provide interface wrappers around AWS GluonTS, but they rely on a complex execution chain passing through reticulate, virtual environments, Python serialization, and MXNet/PyTorch backends. To overcome the performance ...
[Read more...]
1 2 3 … 2,240

Copyright © 2026 | MH Corporate basic by MH Themes