Calculating NNT for indirect comparisons (created 2009-04-20)

This page is moving to a new website.

To calculate the Numbers Needed to Treat (NNT) statistic for response rates when the effect size is shown as an odds ratio I carry out the following calculation:

NNT = (1-(CER*(1-OR))) / ((1-CER)*(CER)*(1-OR)) [1]

CER = Control Event Rate

OR = Odds Ratio

My query occurs when I am calculating this for an indirect comparison. So for example if I am comparing A and B vs a common comparator C I have the following set up:

Indirect comparison gives (for example) A vs B odds ratio of 0.85 (0.6, 1.2). Is it valid to calculate the NNT by substituting CER = 0.8 and OR = 0.85 into the first equation [1]?

You're making this too hard. If you know the two event rates, then the absolute risk reduction (ARR) is simply the difference between the two rates. The number needed to treat (NNT) is simply the inverse of the absolute risk reduction (NNT = 1 / ARR).

The only time you use the more complicated formula that converts the odds ratio to the NNT is when the two event rates are not given to you directly, or if you want to extrapolate the NNT to a different patient who has a higher/lower baseline risk than the average patient in the study you are looking at.

Now you also used the "V" word (valid) and that word always raises my anxiety level. Valid is a loaded word that leads to a lot of heated arguments, partly because it means different things to different people. Let's substitute a different word. Is it "appropriate" to calculate NNT when your control group is effectively a historical control group (all indirect comparisons are comparable to using a historical control group)?

Most researchers only report the NNT for randomized studies. An indirect comparison or a historical control group is a weaker form of evidence, and researchers are reluctant to make the sort of causal statements that would appear in the typical interpretation of NNT.

I'm more laid back than most researchers. It's probably because I'm too old to believe in rigid ideological perspectives. Or maybe I just have gotten too mellow with age.

In any case, I do think you can use the NNT here if you are cautious. You need to bring some additional evidence to bear on the issue.

Generally, evidence from weaker research designs is acceptable if you can show a strong effect, dose response relationship, plausible biological mechanism, replication, and/or other types of corroborating evidence. You're never going to be perfectly happy with this type of data, but in some situations you can still make reasonable inferences. We are fairly confident that smoking causes cancer, even when there are almost no randomized studies about smoking.

I'd also worry a lot about potential confounding variables, as these can cause you to make seriously incorrect inferences if you are not careful.