This page is moving to a new website.
I will be leading an EBM teaching session for housestaff on an article about Methicillin-Resistant Staphylococcus aureus infection rates. I was planning to analyze it using the standard questions about therapy from the Users' Guides to the Medical Literature, but I was wondering if there should be any special considerations, given that therapy (MRSA screening & eradication) was given at a hospital-wide level. For example, the results are presented as incidence of nosocomial MRSA infections per person-years -- can I convert this to a percentage, to churn out a number needed to treat (NNT)? Or is this statistically forbidden? Please let me know of any journal articles you're aware of that address the issue of studies taken at a hospital- or population-based level.
I like the phrase "statistically forbidden". Unfortunately, I don't think there is a "statistics police officer" who could enforce any rules. I'd be glad to take on the role if I could keep 10% of the fines from the statistics citation violations that I write up. Maybe it would be safer to ask, "Is this practice statistically advisable?" If there are no statistics police officers, there are lots of people who offer statistical advice.
I'm unaware of any methodological publications on this issue. I have seen a few examples of number needed to treat (NNT) being calculated on rates, but don't have any particular citations that I can offer. The only thing I can offer is a bit of numeric common sense.
You need to distinguish first between proportions and rates. When you have a fraction where the numerator is a count and the denominator is another count and the numerator represents a subset of the denominator, then you have a proportion. A proportion is guaranteed to be between zero and one and it is unitless.
When you have a fraction where the numerator is a count and the denominator is a measure of time, area, or something in different units than a count, then you have a rate. A rate also occurs if the numerator and the denominator are counts of different units (e.g., number of warranty claims per automobiles sold). A rate always has units, such as deaths per person year of exposure. A rate has the potential of exceeding one, and frequently the units used in the rate are adjusted to make the rate more manageable. Often you will see rates multiplied by one thousand, ten thousand, or a million. So instead of 0.0023 events per person years of exposure, you will see 23 events per ten thousand years of person exposure.
There's an implicit assumption that rates are uniform over time, area or whatever unit appears in the denominator. Otherwise, the rate is difficult to interpret. If infections, for example, occur very frequently on the first day of hospitalization and taper off over time then 24 infections across 500 patients each with a 2 day stay is not directly comparable to 24 infections across 50 patients, each with a 20 day stay.
While it is possible to calculate a NNT for a rate, it should be interpreted with caution. If, for example, a treatment group has an infection rate of 2.4 per thousand patient days and the control group has an infection rate of 2.9 per thousand patient days, then the NNT is two thousand patient days. That means that if you adopted the treatment, you would see on average one fewer event every two thousand patient days. If you have 10 patients on average on a typical day in your unit, then you would see one fewer infection every 200 days on average if you adopted the intervention. Note that this interpretation assumes uniformity across time.
Another issue is that rates are often calculated from observational data, and interpretation of NNT is tricky for observational data. If you note a rate of 2.4 per thousand patient years in females and 2.9 per thousand patient years in males, then the NNT say that you would see one fewer event on average in every 2000 patient days if you could change the sex of all your male patients to female patients. You can't change the sex of your patients, so what does this number really mean?
Another issue that clouds the interpretation of NNT in an observational study is confounding. In your example, comparing unadjusted rates between two hospitals might not make sense if the prognosis is poorer in patients at one hospital because of sociodemographic factors, for example. You could base your calculations on adjusted rates, but this is a rather complex undertaking.
I'm not saying that you can't calculate NNT from observational data, just that you need to be careful about it.