[This article was first published on r on Everyday Is A School Day, and kindly contributed to R-bloggers]. (You can report issue about the content on this page here)
Want to share your content on R-bloggers? click here if you have a blog, or here if you don't.

Here, we have demonstrated three different methods for calculating NNT with meta-analysis data. I learned a lot from this experience, and I hope you find it enjoyable and informative as well. Thank you, @wwrighID, for initiating the discussion and providing a pivotal example by using the highest weight control event proportion to back-calculate ARR and, eventually, NNT. I also want to express my gratitude to @DrToddLee for contributing a brilliant method of pooling a single proportion from the control group for further estimation. Special thanks to @MatthewBJane, the meta-analysis maestro, for guiding me toward the correct equation to calculate event proportions, with weight estimated by the random effect model. đź™Ź

## Prepare Data

library(tidyverse)
library(meta)
library(kableExtra)

df1 <- tibble(study=c("bayliff 1999","pobble 2005","mavs 2006","dipom 2006","bbsa 2007","poise 2008"),event.c=c(0,5,21,2,0,215),n.c=c(50,48,250,459,109,4177))

df2 <- tibble(study=c("bayliff 1999","pobble 2005","mavs 2006","dipom 2006","bbsa 2007","poise 2008"),event.t=c(0,3,19,3,1,152),n.t=c(49,55,246,462,110,4174))

df <- df1 |>
full_join(df2, by = "study")

df |>
kable()

study event.c n.c event.t n.t
bayliff 1999 0 50 0 49
pobble 2005 5 48 3 55
mavs 2006 21 250 19 246
dipom 2006 2 459 3 462
bbsa 2007 0 109 1 110
poise 2008 215 4177 152 4174

Image generated by DALL-E 3

## The Highest Weight Method

One method is find the study with the most weight and narrow 95% confidence interval and use the event proportion of control groups and back-calculate NNT using pooled hazard ratio. This was shown by @wwrightID who posted on an excellent review on how to do this.

For comparison, letâ€™s only focus on the Secure trials

control <- 215/4177
hr <- 0.73
treatment <- hr * control
absolute_risk_reduction <- treatment - control
NNT <- 1/abs(absolute_risk_reduction) # turn negative number to positive

## NNT lower
hr_l <- 0.61
treatment_l <- hr_l * control
arr_l <- treatment_l - control
NNT_l <- ceiling(1/abs(arr_l))

## NNT upper
hr_u <- 0.88
treatment_u <- hr_u * control
arr_u <- treatment_u - control
NNT_u <- ceiling(1/abs(arr_u))

nnt_ci_1 <- paste0(round(NNT)," [95%CI ",NNT_l, "-",NNT_u,"]")


Our control event proportion is 0.0514723. Our calculated treatment event proportion is now 0.0375748. Our NNT with this method is 72 [95%CI 50-162]

## Single Propotion Pooling Method Using Control Group Only

metap <- metaprop(event = event.c, n = n.c, data = df1, studlab = study, method.tau = "ML")

te_ran <- meta:::backtransf(metap$TE.common, sm = "PLOGIT") treatment2 <- hr * te_ran absolute_risk_reduction <- treatment2 - te_ran NNT2 <- 1/abs(absolute_risk_reduction) # turn negative number to positive ## NNT lower hr_l <- 0.61 treatment_l2 <- hr_l * te_ran arr_l2 <- treatment_l2 - te_ran NNT_l2 <- ceiling(1/abs(arr_l2)) ## NNT upper hr_u <- 0.88 treatment_u2 <- hr_u * te_ran arr_u2 <- treatment_u2 - te_ran NNT_u2 <- ceiling(1/abs(arr_u2)) nnt_ci_2 <- paste0(round(NNT2)," [95%CI ",NNT_l2, "-",NNT_u2,"]")  Our control event proportion is now 0.0477125. Our calculated treatment event proportion is now 0.0348302. Our NNT with single proportion pooling method is 78 [95%CI 54-175]. Take note that because Tau2 is very low, essentially means there is no heterogeneity between studies here, hence common model is used. If we had used random effect model here with tau2 of 0 and I2 of 0, our NNT became very large to 200+, give it a try! Too bad metaprop does not provide weights, see discussion here. But the alternative is metap <- metaprop(event = event.c, n = n.c, data = df1, studlab = study, method = "Inverse", method.tau = "DL")  yes! We have weights. But wait, the proportion of event in control group in random effect model is not at all similar to the previous! I think mainly because of different method in calculating tau. Letâ€™s stick with our previous ## Pooling From Full Meta-Analysis Method def <- metabin(event.c = event.c,n.c = n.c, event.e = event.t, n.e = n.t, studlab = study,data = df,sm="RR",level = 0.95,comb.fixed=T,comb.random=T,hakn = F) summary(def)  #### Can We Reproduce Same Forest Plot? forest(def)  Not too shabby! Weights did not appear to be similar but the hazard ratio looks very similar, same with the 95% CI. Notice that there is lack of heterogeneity between studies, hence the fixed and random effect models are very similar. Letâ€™s dive into calculating NNT with our last technique. But how? # new dataframe with newly assigned weights weights <- def$w.random / sum(def$w.random) df_new <- df |> add_column(weights = weights) |> mutate(total_weights = sum(weights), log_t = log(event.t/n.t)*weights, log_t = case_when( is.infinite(log_t) ~ log(0.5/n.t)*weights, # Haldane-Anscombe correction TRUE ~ log_t )) |> mutate(log_c = log(event.c/n.c)*weights, log_c = case_when( is.infinite(log_c) ~ log(0.5/n.c)*weights, TRUE ~ log_c )) |> drop_na() total_weights <- sum(df_new$weights)

# average event prop on treatment
prop_t <- exp(sum(df_new$log_t) / total_weights) # average event prop control prop_c <- exp(sum(df_new$log_c) / total_weights)

# RR random effect?
rr <- prop_t/prop_c

# arr
absolute_risk_reduction <- prop_t - prop_c

# NNT
NNT3 <- 1/abs(absolute_risk_reduction)

# NNT lower
var_arr <- prop_t * (1-prop_t) / sum(df_new$n.t) + prop_c * (1-prop_c) / sum(df_new$n.c)
nnt_l3 <- ceiling(1/abs(absolute_risk_reduction - 1.96*sqrt(var_arr)))

# NNT upper
nnt_u3 <- ceiling(1/abs(absolute_risk_reduction + 1.96*sqrt(var_arr)))

nnt_ci_3 <- paste0(ceiling(NNT3)," [95%CI ",nnt_l3, "-",nnt_u3,"]")


Our control event proportion is now 0.0528544. Our calculated treatment event proportion is now 0.0386394. Our NNT with single proportion pooling method is 71 [95%CI 45-165]. Very close to our first method! Mainly because the weights given on our random effect model highly favors poise 2018, it makes sens that our control event proportion should be quite similar to such!

The equation behind calculating the proportion for both treatment and control via its weight is:
$$\begin{gather} \ln(\text{prop}_t) = \sum \ln(\text{prop}_{it}) \cdot \text{weight}_{it} \\ \text{prop}_t = e^{\sum \ln(\text{prop}_{it}) \cdot \text{weight}_{it}} \end{gather}$$

$$prop_t$$: overall proportion of treatment or control group.
$$prop_{it}$$: proportion of ith study of treatment or control group (e.g,bbsa, poise).
$$weight_{it}$$: random effect weights of ith study of treatment or control group.

#### How To Calculate NNT Lower and Upper Bound?

$$\begin{gather} \text{ARR} = \text{prop}_t - \text{prop}_c \\ var(\text{ARR}) = \frac{\text{prop}_t \cdot (1-\text{prop}_t)}{\text{n}_t} + \frac{\text{prop}_c \cdot (1-\text{prop}_c)}{\text{n}_c} \\ \text{ARR 95\% CI} = \text{ARR} \pm 1.96 \cdot \sqrt{\text{var}(\text{ARR})} \end{gather}$$

ARR: Absolute Risk Reduction.
$$var$$: Variance.
$$prop_t$$: Pooled Event Proportion in Treatment Group.
$$prop_c$$: Pooled Event Proportion in Control Group.
CI: Confidence Interval, 95% preferred here, hence the z score of 1.96 used

## Comparison of All 3 Methods

df_compare <- tibble(method=c("highest_weight","single_prop_pool","full"),prop_control=c(control,te_ran,prop_c),prop_treatment=c(treatment,treatment2,prop_t),nnt=c(nnt_ci_1,nnt_ci_2,nnt_ci_3),tau2=rep(def$tau2,3),I2=rep(def$I,3))

df_compare |>
kable()

method prop_control prop_treatment nnt tau2 I2
highest_weight 0.0514723 0.0375748 72 [95%CI 50-162] 0 0
single_prop_pool 0.0477125 0.0348302 78 [95%CI 54-175] 0 0
full 0.0528544 0.0386394 71 [95%CI 45-165] 0 0

## Exploring Other Meta-analysis Data & Compare The 3 Methods

#### Methenamine for Preventing UTI Without Renal Tract Abnormalities

Using furness 1975 as our control given highest weight and narrow CI.

method prop_control prop_treatment nnt tau2 I2
highest_weight 0.2537313 0.0659701 5 [95%CI 5-21] 0.8150958 0.6551118
single_prop_pool 0.2147027 0.0558227 6 [95%CI 6-25] 0.8150958 0.6551118
full 0.2174201 0.0558707 7 [95%CI 5-10] 0.8150958 0.6551118

Not too shabby! All 3 methods are quite similar in this heterogenous, low n studies. However, full method has a more narrow 95%CI of NNT. Interesting!

#### Multidisciplinary Teams for the Management of Infective Endocarditis: A Systematic Review and Meta-analysis

Short-term mortality of patients with infective endocarditis

Using Diab 2021 as our control given highest weight (10%) through random effect.

method prop_control prop_treatment nnt tau2 I2
highest_weight 0.2567237 0.1540342 10 [95%CI 8-17] 0.1465014 0.62777
single_prop_pool 0.2456502 0.1473901 10 [95%CI 8-18] 0.1465014 0.62777
full 0.2428684 0.1448027 11 [95%CI 9-14] 0.1465014 0.62777

All 3 methods appear to have very similar NNT, the only difference is 95% CI of NNT.

#### Effect of Corticosteroids on Mortality and Clinical Cure in Community-Acquired Pneumonia: A Systematic Review, Meta-analysis, and Meta-regression of Randomized Control Trials

Saleem N Chest. 2023 Mar;163(3):484-497. doi: 10.1016/j.chest.2022.08.2229..

Using Blum 2015 with highest weight 38.5%.

method prop_control prop_treatment nnt tau2 I2
highest_weight 0.0550000 0.0363000 53 [95%CI 34-607] 0 0
single_prop_pool 0.0515075 0.0339949 57 [95%CI 36-648] 0 0
full 0.0559733 0.0368395 53 [95%CI 29-329] 0 0

Both highest weight and full method have similar NNT. Slightly different for single proportion, but all quite similar for estimation. NNT lower and upper bound on this example has dramatic difference when full method is compared with highest weight and single proportion.

## Acknowledgement

Thank you, @wwrighID, for initiating the discussion and providing a pivotal example by using the highest weight control event proportion to back-calculate ARR and, eventually, NNT. I also want to express my gratitude to @DrToddLee for contributing a brilliant method of pooling a single proportion from the control group for further estimation. Special thanks to @MatthewBJane, the meta-analysis maestro, for guiding us toward the correct equation to calculate event proportions, with weight estimated by the random effect model.

This discussion has been incredibly inspirational and educational! A heartfelt thank you to everyone involved for helping me answer a question that has intrigued me since 2021.

## Opportunity for Improvement

• I am not certain if the 95%CI equation for both highest weight and single proportion is accurate. It made sense for the highest weight to use RR lower and upper bound and back calculate, but for the single proportion the metaprop actually provides a 95% interval for the proportion itself, should we than use these lower and upper bound and fix RR to back calculate treatment? đź¤·â€Ťâ™‚ď¸Ź If you do, please leave a comment below
• Tackle odds ratio next. There are some great meta-analysis studies out there with OR as estimates. Shouldnâ€™t be too hard, just need to be mindful about odds calculation and its more tedious conversion to proportion which eventually will need to calculate ARR and NNT.
• If you spot any mistakes, please feel free to send me a message, Iâ€™d love to learn from it!

## Lessons learnt

• Learnt how to estimate NNT from all 3 methods (highest weight, single proportion pooling, full)
• I made an Rscript of the function called meta_compare using all 3 methods if youâ€™re interested
• Learnt how to calculate NNT lower and upper bound through variance of ARR
• Have to use meta:::backtransf to convert random effect estimate
• Learnt \pm is $$\pm$$ in latex, and add \ in front of % in \text{}
• Calculating pooled treatment and control group with its weight estimated by random effect model
• Finally, this answered my question I had since 2021, how to calculate NNT with meta-analysis!