P.Mean >> Category >> Early stopping in clinical trials (created 2007-06-18).

Clinical trials are sometimes stopped because of early evidence of efficacy, early evidence of harm, or early evidence of futility. In general, the rules for stopping a study need to be specified in the research protocol before any data is collected. These pages discuss some of the issues associated with early stopping of clinical trials. Articles are arranged by date with the most recent entries at the top. Closely related pages can be found at Category: Hypothesis testing, Category: Randomization in research, and Category: Sample size justification. You can find outside resources at the bottom of this page.

2011

17. P.Mean: Net-accessible resources for group sequential designs (created 2011-10-19). Dear Professor Mean: I am trying to locate a good net-accessible resource for the group sequential designs, and sample size re-estimation for adaptive designs. Can you help?

2009

16. The Monthly Mean: Can I stop this study early? (September/October 2009)

15. What IRBs should look for in a data safety and monitoring plan. (February 2009)

2008

14. P.Mean: Can I stop this study? (created 2009-05-29). I got an email from a researcher on a project I was peripherally involved with awhile back. Here's what she wrote (with a few details removed to protect anonymity). As you all are aware, enrollment for the BLANK study has been slower than anticipated. However, due to a high suspicion that patients in the CONTROL arm were having more complications (more rescue therapy) and less improvement, we have decided to look at the data prior to reaching our initial proposed N=140. We had 79 patients enrolled. We found that significantly more patients in the TREATMENT arm reported that their main symptom was better at 24 hours than the CONTROL arm (p=0.02). Also, we had 6 patients need some kind of rescue, 5 of those were in the CONTROL arm (this approached statistical significance, p=0.08). Therefore, I am writing to see if you agree with stopping the study at this point. Please let me know at your earliest convenience.

13. P.Mean: What is the Lan-DeMets approach to interim analysis? (created 2008-11-21). I read an article that talked about a trial that ended early. They describe the approach as a "O'Brien-Fleming stopping boundaries determined by means of the Lan-DeMets approach.". Does anyone you know anything about this statistical technique to determine if this is a valid approach?

Other resources:

Adaptive, group sequential and decision theoretic approaches to sample size determination. C. R. Mehta, N. R. Patel. Stat Med 2006: 25(19); 3250-69; discussion 3297-301, 3302-4, 3313-4, 3326-47. [Medline]

Adaptive vs Group Sequential Designs: Some Trade-Offs [pdf]. Cyrus R. Mehta, Anastasios A. Tsiatis, Cytel Statistical Software. Accessed on 2003-07-15. www.cytel.com/papers/ENARTalk.pdf

Design and Interim Monitoring of Flexible Clinical Trials [pdf]. Mehta CR, Cytel Software. Accessed on 2004-03-25. www.cytel.com/Papers/MBC_2004.pdf

Biopharmaceutical Sequential Statistical Applications. Karl E. Peace (1992) New York: Marcel Dekker, Inc.

Data safety and monitoring boards. Arthur S Slutsky, James V Lavery. NEJM 2004: 350(11); 1143-1147.

Data And Safety Monitoring Plans. UCSF School of Medicine. Accessed on 2003-01-21. As of October, 2000, the NIH began requiring Data and Safety Monitoring plans (DSMPs) for all NIH funded clinical trials. For more information regarding this requirement and how to process these plans at UCSF, see the Data and Safety Monitoring Plans announcement on the Contracts and Grants webpage. Below are templates to assist you in preparing DSMPs when you have received a formal notification from NIH to submit a final DSMP. These templates are meant to guide you through the issues to think about when writing a DSMP. BROKEN LINK: http://www.research.ucsf.edu/cg/memo/data_safety.htm

Design and analysis of prostate cancer trials. Sylvester R. Acta Urologica Belgica 1994: 62(1); 23-29.

John Whitehead. The Design and Analysis of Sequential Clinical Trials 2nd ed. Wiley; 1997.

Design and Interim Monitoring of Flexible Clinical Trials [pdf]. Cyrus R. Mehta, Presented at Massachusetts Biotechnology March 9, 2004 - Boston, MA. Accessed on 2004-03-25. www.cytel.com/Papers/MBC_2004.pdf

Ethical dilemmas in continuing a zidovudine trial after early termination of similar trials. Simberkoff MS, Hartigan PM, Hamilton JD, Deykin D, Gail M, Bartlett JG, Feorino P, Redfield R, Roberts R, Collins D, et al. Control Clin Trials 1993: 14(1); 6-18.

The Ethics of Early Stopping Rules: Who Is Protecting Whom? Cannistra SA. Journal of Clinical Oncology 2004: 22(9); 1542-1545. [PDF]

FDA Guidance on Clinical Trial Data Monitoring Committees (DMCs). U.S. Food and Drug Administration, Transcript of an open public meeting, November 27, 2001. Accessed on 2003-05-12. Good morning. On behalf of FDA I'd like to welcome you to today's workshop on data monitoring committees. The purpose of this workshop is to introduce FDA's new guidance for clinical trial sponsors on the establishment and operation of clinical trial data monitoring committees. We planned this workshop several months ago with the expectation certainly that this guidance document would be out with ample time for individuals to review it in advance of the workshop. We may not have had quite as much time for this review process as we would have hoped but we are very pleased to at least see that the document is available and is, in fact, available for general circulation today outside. BROKEN LINK: www.fda.gov/cber/minutes/clinmon1101p1.htm

Flexible Clinical Trial Design [pdf]. Cyrus Mehta, Presented at the Adaptive Design For Clinical Trials: Statistical Methodologies and Applications Conference, March 4, 5, 2004 - Philadelphia, PA. Accessed on 2004-03-25. www.cytel.com/Papers/flexible_ct_design_04.pdf

Further Guidance on a Data and Safety Monitoring for Phase I and Phase II Trials. U.S. National Institutes of Health. Accessed on 2003-05-12. In June 1998, the National Institutes of Health (NIH) issued a policy on data and safety monitoring that requires oversight and monitoring of all intervention studies to ensure the safety of participants and the validity and integrity of the data. The policy further elaborates that monitoring should be commensurate with risks and with the size and complexity of the trials. The NIH already requires data and safety monitoring, generally, in the form of Data and Safety Monitoring Boards (DSMBs) for phase III clinical trials. For earlier trials (phase I and II), a DSMB may be appropriate if the studies have multiple clinical sites, are blinded (masked), or employ particularly high-risk interventions or vulnerable populations. grants1.nih.gov/grants/guide/notice-files/NOT-OD-00-038.html

Group Sequential Methods with Applications to Clinical Trails. Christopher Jennison, Bruce W. Turnbull (2000) Boca Raton, Florida: Chapman & Hall/CRC.

Group sequential designs for monitoring survival probabilities. Lin DY, Shen L, Ying Z, Breslow NE. Biometrics 1996: 52(3); 1033-41.

Guidance for Clinical Trial Sponsors. On the Establishment and Operation of Clinical Trial Data Monitoring Committees. U.S. Food and Drug Administration. Accessed on 2003-05-12. This guidance discusses the roles, responsibilities and operating procedures of Data Monitoring Committees (DMCs) that carry out important aspects of clinical trial monitoring. The document is intended to assist sponsors of clinical trials in determining when a DMC is needed for optimal study monitoring, and how such committees should operate. We recognize that in many clinical trials the sponsor delegates much decision-making regarding the design and conduct of the trial to some other entity such as a Steering Committee (see Section 3.2) or Contract Research Organization (CRO) (see 21 CFR 312.3(b)). In this document, references to the sponsor with regard to trial management and decision-making should be understood to refer also to any individual or group to which the sponsor has delegated the relevant management responsibilities. BROKEN LINK: www.fda.gov/cber/gdlns/clindatmon.htm

Human Subjects Case 2: Early Termination of Clinical Trials. Michael Kalichman, Susan Eastwood. Accessed on 2003-06-20. Dr. Browning, entering his fellowship in pulmonary medicine, is assigned to carry out an institutional review board (IRB)-approved study designed by Dr. Sedgwick. The study of patients with cystic fibrosis is designed to determine whether an experimental drug improves pulmonary function and reduces the incidence of new infections. The study is double-blinded, and the patients receive inhalers with either placebo or the experimental drug. Preclinical studies were promising, but potential problems had included an allergic reaction to the drug and a direct, adverse chemical effect on the lung passages. ethics.ucsd.edu/workshops/CRW/cases/case2.htm

Interim analyses of data as they accumulate in laboratory experimentation. Ludbrook J. BMC Med Res Methodol 2003: 3(1); 15. [Medline] [Abstract] [Full text] [PDF]

Issues in data monitoring and interim analysis of trials. AM Grant, DG Altman, AB Babiker, MK Campbell, FJ Clemens, JH Darbyshire, DR Elbourne, SK McLeer, MKB Parmar, SJ Pocock, DJ Spiegelhalter, MR Sydes, AF Walker, SA Wallace. Health Technolgoy Assessment 2005: 9(7); [Medline] [Abstract] [PDF]

Monitoring and ensuring safety during clinical research. M. A. Morse, R. M. Califf, J. Sugarman. Jama 2001: 285(9); 1201-5. [Medline] [Abstract]

Monitoring clinical trials--interim data should be publicly available. Lilford RJ, Braunholtz D, Edwards S, Stevens A. British Medical Journal 2001: 323(7310); 441-2. [Medline] [Full text] [PDF]

Monitoring randomised controlled trials. Parkinson's disease trial illustrates the dangers of stopping early. Abrams KR. British Medical Journal 1998: 316(7139); 1183-4. [Medline] [Full text]

NIH Policy for Data and Safety Monitoring. U.S. National Institutes of Health. Accessed on 2003-05-12. It is the policy of the NIH that each Institute and Center (IC) should have a system for the appropriate oversight and monitoring of the conduct of clinical trials to ensure the safety of participants and the validity and integrity of the data for all NIH-supported or conducted clinical trials. The establishment of the data safety monitoring boards (DSMBs) is required for multi-site clinical trials involving interventions that entail potential risk to the participants. The data and safety monitoring functions and oversight of such activities are distinct from the requirement for study review and approval by an Institutional Review Board (IRB). grants1.nih.gov/grants/guide/notice-files/not98-084.html

Optimal three-stage designs for phase II cancer clinical trials. Chen TT. Stat Med 1997: 16(23); 2701-11. [Medline]

Premature discontinuation of clinical trial for reasons not related to efficacy, safety, or feasibility Commentary: Early discontinuation violates Helsinki principles. Lievre M, Menard J, Bruckert E, Cogneau J, Delahaye F, Giral P, Leitersdorf E, Luc G, Masana L, Moulin P, Passa P, Pouchain D, Siest G, Boyd K. BMJ 2001: 322(7286); 603-606. [Full text] [PDF]

Reporting Adverse Event/Safety Issues. Penn State College of Medicine. Accessed on 2004-01-28. BROKEN LINK: www.hmc.psu.edu/hmc-irb/Forms%20&%20Instructions/Ongoing%20Studies/OngoingStudies.htm

Responsibilities of sponsors are limited in premature discontinuation of trials. Ashcroft R. BMJ 2001: 323(7303); 53-. [Full text]

Societal responsibilities of clinical trial sponsors. Evans S, Pocock S. BMJ 2001: 322(7286); 569-570. [Medline] [Full text] [PDF]

Safeguarding patients in clinical trials with high mortality rates. Freeman BD, Danner RL, Banks SM, Natanson C. Am J Respir Crit Care Med 2001: 164(2); 190-192. [Medline] [Full text] [PDF]

Sample size and interim analysis issues for dose selection. Lee KL. Am Heart J 2000: 139(4); S161-5. [Medline]

Selegiline, or the problem of early termination of clinical trials. The clinical questions are not well answered, and probably never will be. Breteler MM. British Medical Journal 1998: 316(7139); 1182-3. [Medline] [Full text]

Sequential Methods in Statistics. Wetherill BG, Glazebrook KD (1986) New York: Chapman and Hall.

A sequential procedure for monitoring clinical trials against historical controls. X. Xiong, M. Tan, J. Boyett. Stat Med 2006; [Medline]

Should all trials have a data safety and monitoring committee? Cairns JA, Hallstrom A, Held P. Am Heart J 2001: 141(1); 156-63. [Medline]

Tamoxifen for the prevention of breast cancer. Important questions remain unanswered, and existing trials should continue. Bruzzi P. British Medical Journal 1998: 316(7139); 1181-2. [Medline] [Full text]

Vasectomy by ligation and excision, with or without fascial interposition: a randomized controlled trial [ISRCTN77781689]. D Sokal, B Irsula, M Hays, M Chen-Mok, MA Barone. BMC Med 2004: 2(1); 6. [Medline] [Abstract] [Full text] [PDF]

When and how to stop a clinical trial: invited remarks. Chalmers TC. Clin Pharmacol Ther 1979: 25(5 Pt 2); 649-50. [Medline]

Examples

Breast cancer trial stopped early [news]. Josefson D. British Medical Journal 1998: 316(7139); 1185. [Medline] [Full text]

Lessons learned from a clinical trial. P. W. Armstrong , L. K. Newby, C. B. Granger, K. L. Lee, R. J. Simes, F. Van de Werf, H. D. White, R. M. Califf. Circulation 2004: 110(23); 3610-4. [Medline] [Full text] [PDF]

Patients at high risk of death after lung-volume-reduction surgery. National Emphysema Treatment Trial Research Group. N Engl J Med 2001: 345(15); 1075-83. [Medline] [Abstract]

Risks and benefits of estrogen plus progestin in healthy postmenopausal women: principal results from the Women's Health Initiative randomized controlled trial. Rossouw JE, Anderson GL, Prentice RL, LaCroix AZ, Kooperberg C, Stefanick ML, Jackson RD, Beresford SAA, Howard BV, Johnson KC, Kotchen JM, Ockene J. Jama 2002: 288(3); 321-33. [Medline] [Abstract]

Study of Asthma-Drug Halted. U.S. Food and Drug Administration. Accessed on 2003-01-29. The Food and Drug Administration (FDA) today announced that an interim analysis of a large safety study of the approved asthma drug Serevent (salmeterol xinafoate) Inhalation Aerosol suggests that the drug may be associated with an increased risk of life-threatening asthma episodes or asthma-related deaths, particularly in some patients. The interim analysis did not show a statistically significant result for the primary endpoint - a combination of respiratory-related deaths or intubations (or ventilatory failure). There was a trend, however, towards increases in asthma deaths and serious asthma episodes when all patients in the study were considered, though again this did not reach statistical significance. A further analysis of the data from the study suggested that the risk might be greater in African-American patients. Also, further analyses showed that patients not taking inhaled corticosteroids at study entry appeared to have greater risk for serious outcomes than those who were taking inhaled corticosteroids. BROKEN LINK: www.fda.gov/bbs/topics/ANSWERS/2003/ANS01192.html

Upper Extremity Deep Venous Thrombosis in Cancer Patients with Venous Access Devices- Prophylaxis with a Low Molecular Weight Heparin (Fragmin). Monreal M, Alastrue A, Rull M, Mira X, Muxart J, Rosell R, Abad A. Thromb Haemost 1996: 75(2); 251-253. [Medline]

All of the material above this paragraph is licensed under a Creative Commons Attribution 3.0 United States License. This page was written by Steve Simon and was last modified on 2017-06-15. The material below this paragraph links to my old website, StATS. Although I wrote all of the material listed below, my ex-employer, Children's Mercy Hospital, has claimed copyright ownership of this material. The brief excerpts shown here are included under the fair use provisions of U.S. Copyright laws.

2008

12. Stats: What does one-third of the way through a study mean? (April 6, 2008). Someone asked me a very good question regarding interim analysis. If the call for an interim analysis is specified as occurring one-third (and two-third) of the way through a study, what does that mean. In a study with 60 patients lasting a full year, does that mean until 20 (40) patients  have arrived, or does it mean until 4 (8) months? Also, if you are counting discrete events, such as adverse events, does it mean one-third of the expected number of adverse events?

2006

11. Stats: When are two events worrisome? (December 4, 2006). I was attending a workshop on developing a risk management plan for a new drug. The presenter, Nawab Qizilbash from Oxon Clinical Epidemiology, gave an interesting example. Suppose you are running a clinical trial with 50 total patients. In the treatment group, you notice two adverse events and in the placebo group you notice zero adverse events. Should you stop the trial?

10. Stats: Oops! I found statistical significance in my pilot study (September 18, 2006). Dear Professor Mean, I ran a pilot study to help me estimate the sample size for a larger study. There were 15 patients in each arm of the pilot, and when I analyzed the pilot data for efficacy in the primary outcome, I was shocked to find out that the results were already statistically significant. Do I still need to run the full study?

9. Stats: Stopping rules for a Data Safety Monitoring Plan (July 5, 2006). If a research study requires a DSMP (Data Safety Monitoring Plan), that plan should outline conditions that would cause a study to end early. It is difficult to specify what those conditions would be, but it is important to at least think about and comment on each of the major areas listed below.

2005

8. Stats: Group Sequential Monitoring of Clinical Trials in R (December 13, 2005). It is very expensive to purchase software that performs group sequential monitoring of clinical trials (sometimes called interim analysis). Group sequential monitoring is looking at a trial at selected time points during the study to see if you should stop the study early. There are a couple of functions in R that will do simple calculations, and the price, of course, is free.

7. Stats: When can I stop my CQI study? (June 6, 2005). I was asked today about a CQI project where children were being tested for a certain condition, but they never tested positive. This was going on for over a year and there were already 188 cultures and none were positive. Is it safe to hang up your spurs and call it a day?

6. Stats: Do I have enough data after 24 months of time? (April 5, 2005). Someone asked me about a correlation coefficient that he computed on a data set that represented 24 months of data collection. A particular correlation of interest (a correlation between staff turnover and resident falls) was not significantly different from zero, but this person wanted to know how much more data to collect before safely concluding that no relation has been or likely will be established. First compute a confidence interval for the correlation coefficient. If that interval is so narrow that you can rule out the possibility of a clinically important shift, then your sample size is large enough.

2004

5. Stats: Controversy over stopping a study early (November 24, 2004). A while back, the IRB asked me to look into a randomized study where the interim report indicated a huge disparity in the two treatment arms. One arm of the study had almost all good outcomes and the other arm had almost all bad outcomes or at best no improvement. The sample size, though, was only 20 patients, and the protocol had no formal rule for stopping the study early. Even without such a rule, a careful analysis of the data revealed that there was little justification for continuing randomization when one arm of the study was clearly inferior.

4. Stats: Early stopping in an animal experiment (July 1, 2004). I was helping someone with power calculations for an animal experiment. There were several ways to estimate power and they all seemed to point to the use of either 8 animals per group or 12 animals per group. I explained that if she was more risk averse, she should use 12 animals and if she were more work averse, she should use 8 animals. Her answer surprised me a bit. She said that she liked the idea of using 12 animals per group. That way, if the results achieved significance after only 8 animals, she could go back to the Animal Care and Use Committee and tell that she saved a few animals.

2002

3. Stats: Stopping a study early (October 29, 2002). Dear Professor Mean, I tried really hard to recruit the number of subjects that I promised to in my power calculations, but I just can't do it. I'm thinking about stopping the study early, but I'm worried that it might screw up all my statistics. -- Exhausted Evelyn

2000

2. Stats: Protocol changes (December 22, 2000). Dear Professor Mean, After I collected my data, I noticed a problem that I had not anticipated. I want to make some protocol changes and analyze my data differently. Can I do this?

1999

1. Stats: Interim analysis (September 13, 1999). Dear Professor Mean, I'm going on a job interview and I know one of the questions they will ask me is about interim analysis. What should I tell them? -- Harried Howard

What now?

Browse other categories at this site

Browse through the most recent entries

Get help

This work is licensed under a Creative Commons Attribution 3.0 United States License. This page was written by Steve Simon and was last modified on 2017-06-15.