This page is moving to a new website.
I got an email from a researcher on a project I was peripherally involved with awhile back. Here's what she wrote (with a few details removed to protect anonymity).
As you all are aware, enrollment for the BLANK study has been slower than anticipated. However, due to a high suspicion that patients in the CONTROL arm were having more complications (more rescue therapy) and less improvement, we have decided to look at the data prior to reaching our initial proposed N=140. We had 79 patients enrolled. We found that significantly more patients in the TREATMENT arm reported that their main symptom was better at 24 hours than the CONTROL arm (p=0.02). Also, we had 6 patients need some kind of rescue, 5 of those were in the CONTROL arm (this approached statistical significance, p=0.08). Therefore, I am writing to see if you agree with stopping the study at this point. Please let me know at your earliest convenience.
I am always willing to offer an opinion, but first I wanted to confirm that there was no official guidelines for early stopping in the protocol that was submitted to the IRB and that there was no formal data safety and monitoring committee. If that turns out to be the case, then any choice made at this point will be somewhat arbitrary from a statistical perspective. There is no magic number that can be calculated post hoc that will provide definitive guidance here.
First and foremost, you need to acknowledge that the primary investigator has the right (and the duty) to stop a study if he/she a strong gut feeling that continuing the study is bad for the patients enrolling in the study. I would never use a statistical argument to overrule someone's gut feeling, though I would be glad to discuss whether the data tends to support their gut or not. Early stopping often involves a tradeoff between patient safety and scientific integrity, and I would almost always side with patient safety.
I would not let failure to pre-specify stopping rules be a barrier either. It's a bad idea for a researcher to fail to anticipate and document in the protocol reasons for early stopping, but we should not punish future patients because of this oversight. There is an implicit stopping rule that exists for every protocol, whether it is stated or not. Any research protocol can be terminated if, in the opinion of the principle investigator, further continuance of the study would put some or all of the future patients at an unnecessary level of risk. The principal investigator does not need the approval of a statistician, the IRB, or any other party to invoke this rule, though consultation with various people is always reasonable.
If I were asked about post hoc justification for early stopping, I would try first to create a plausible set of stopping rules that was as independent of the current data analysis as possible. This is impossible to do perfectly, but a good standard for post hoc justification is any rule that would have been likely to have been received well if it had been proposed during the protocol development stage.
Finally, stopping a study permanently is a pretty serious decision, but I certainly would suggest that a temporary freeze of the study (no enrollment of new patients) is a safe and judicious action to take while discussions are ongoing.