|
|
P.Mean >>
Category >> Pilot studies (created
2007-08-12).
|
Pilot studies are research studies which produce no direct benefit to the research community, but rather which provide benefit through assisting with the planning of a future research study. These pages present some of the issues associated with running a pilot study. Also see Category: Survey design.
2010
5. The Monthly Mean: Internal pilot study (December 2010)
4. P.Mean: Pilot study (created 1999-09-03, updated 2010-07-08). Dear Professor Mean, I am proposing a research study that will examine a complex intervention of diet, exercise, and behavioral modification for some of my pediatric patients who need to lose weight. I want to collect some data from a pilot study before I start the research study. How do I describe the pilot study in my protocol? -- Sophisticated Sarah
Journal article: Milensu Shanyinde, Ruth Pickering, Mark Weatherall. Questions asked and answered in pilot and feasibility randomized controlled trials BMC Medical Research Methodology. 2011;11(1):117. Abstract: "BACKGROUND: In the last decade several authors have reviewed the features of pilot and feasibility studies and advised on the issues that should be addressed within them. We extend this literature by examining published pilot/feasibility trials that incorporate random allocation, examining their stated objectives, results presented and conclusions drawn, and comparing drug and non-drug trials. METHODS: A search of EMBASE and MEDLINE databases for 2000 to 2009 revealed 3652 papers that met our search criteria. A random sample of 50 was selected for detailed review. RESULTS: Most of the papers focused on efficacy: those reporting drug trials additionally addressed safety/toxicity; while those reporting non-drug trials additionally addressed methodological issues. In only 56% (95% confidence intervals 41% to 70%) were methodological issues discussed in substantial depth, 18% (95% confidence interval 9% to 30%) discussed future trials and only 12% (95% confidence interval 5% to 24%) of authors were actually conducting one. CONCLUSIONS: Despite recent advice on topics that can appropriately be described as pilot or feasibility studies the large majority of recently published papers where authors have described their trial as a pilot or addressing feasibility do not primarily address methodological issues preparatory to planning a subsequent study, and this is particularly so for papers reporting drug trials. Many journals remain willing to accept the pilot/feasibility designation for a trial, possibly as an indication of inconclusive results or lack of adequate sample size." [Accessed on September 3, 2011]. http://www.biomedcentral.com/1471-2288/11/117.
M Kieser, T Friede. Re-calculating the sample size in internal pilot study designs with control of the type I error rate. Stat Med. 2000;19(7):901-911. Abstract: "When designing a clinical trial, there is usually some uncertainty about the variability of the primary outcome variable. This may lead to an unnecessarily high or inadequately low sample size. The internal pilot study approach uses data from patients recruited up to an interim stage to re-estimate the variance and to re-calculate the final sample size accordingly. Previously, simulation studies have shown that this methodology may highly improve the chance to obtain a well-powered trial. However, it also turned out that the type I error rate may be inflated by this procedure. We quantify the maximum excess of the type I error rate for normally distributed outcomes. If strict control of the alpha-level is considered to be an important issue, a method is proposed to achieve this when re-calculating the sample size in internal pilot studies. The characteristics of the power distributions are investigated for various sample size adaptation rules and implications are discussed." [Accessed December 20, 2010]. Available at: http://www.ncbi.nlm.nih.gov/pubmed/10750058.
J Wittes, E Brittain. The role of internal pilot studies in increasing the efficiency of clinical trials. Stat Med. 1990;9(1-2):65-71; discussion 71-72. Abstract: "Investigators often design clinical trials without knowing precisely the values of such necessary parameters as the variances or the event rates in the control group. In order to determine reasonable values for such parameters, they may design a small pilot study external to the main trial. In this paper we propose designs, which we term internal pilot studies, that designate a portion of the main trial as a pilot phase. At the end of the internal pilot study, the investigators recompute preselected parameters and recalculate required sample size. The study then proceeds with the modifications dictated by the internal pilot. Final analyses of the results incorporate all data, disregarding the fact that part of the data came from a pilot phase. As one example of this type of design, we consider a study to compare two normally distributed means. By simulation, we show a numerical example for which the effect of the procedure on the alpha-level is negligible, but the potential gain in power considerable. We urge considering a similar approach for a number of types of endpoints." [Accessed December 20, 2010]. Available at: http://www.epibiostat.ucsf.edu/courses/RoadmapK12/SIGS/Wittes.pdf.
Tim Friede, Meinhard Kieser. Sample size recalculation in internal pilot study designs: a review. Biom J. 2006;48(4):537-555. The adequacy of sample size is important to clinical trials. In the planning phase of a trial, however, the investigators are often quite uncertain about the sizes of parameters which are needed for sample size calculations. A solution to this problem is mid-course recalculation of the sample size during the ongoing trial. In internal pilot study designs, nuisance parameters are estimated on the basis of interim data and the sample size is adjusted accordingly. This review attempts to give an overview on the available methods. It is written not only for biometricians who are already familar with the the topic and wish to update their knowledge but also for users new to the subject. [Accessed December 20, 2010]. Available at: http://www.ncbi.nlm.nih.gov/pubmed/16972704.
Lehana Thabane, Jinhui Ma, Rong Chu, et al. A tutorial on pilot studies: the what, why and how. BMC Medical Research Methodology. 2010;10(1):1. Abstract: "Pilot studies for phase III trials - which are comparative randomized trials designed to provide preliminary evidence on the clinical efficacy of a drug or intervention - are routinely performed in many clinical areas. Also commonly know as "feasibility" or "vanguard" studies, they are designed to assess the safety of treatment or interventions; to assess recruitment potential; to assess the feasibility of international collaboration or coordination for multicentre trials; to increase clinical experience with the study medication or intervention for the phase III trials. They are the best way to assess feasibility of a large, expensive full-scale study, and in fact are an almost essential pre-requisite. Conducting a pilot prior to the main study can enhance the likelihood of success of the main study and potentially help to avoid doomed main studies. The objective of this paper is to provide a detailed examination of the key aspects of pilot studies for phase III trials including: 1) the general reasons for conducting a pilot study; 2) the relationships between pilot studies, proof-of-concept studies, and adaptive designs; 3) the challenges of and misconceptions about pilot studies; 4) the criteria for evaluating the success of a pilot study; 5) frequently asked questions about pilot studies; 7) some ethical aspects related to pilot studies; and 8) some suggestions on how to report the results of pilot investigations using the CONSORT format." [Accessed July 12, 2010]. Available at: http://www.biomedcentral.com/1471-2288/10/1.
Mubashir Arain, Michael Campbell, Cindy Cooper, Gillian Lancaster. What is a pilot or feasibility study? A review of current practice and editorial policy. BMC Medical Research Methodology. 2010;10(1):67. Abstract: "BACKGROUND: In 2004, a review of pilot studies published in seven major medical journals during 2000-01 recommended that the statistical analysis of such studies should be either mainly descriptive or focus on sample size estimation, while results from hypothesis testing must be interpreted with caution. We revisited these journals to see whether the subsequent recommendations have changed the practice of reporting pilot studies. We also conducted a survey to identify the methodological components in registered research studies which are described as 'pilot' or 'feasibility' studies. We extended this survey to grant-awarding bodies and editors of medical journals to discover their policies regarding the function and reporting of pilot studies. METHODS: Papers from 2007-08 in seven medical journals were screened to retrieve published pilot studies. Reports of registered and completed studies on the UK Clinical Research Network (UKCRN) Portfolio database were retrieved and scrutinized. Guidance on the conduct and reporting of pilot studies was retrieved from the websites of three grant giving bodies and seven journal editors were canvassed. RESULTS: 54 pilot or feasibility studies published in 2007-8 were found, of which 26 (48%) were pilot studies of interventions and the remainder feasibility studies. The majority incorporated hypothesis-testing (81%), a control arm (69%) and a randomization procedure (62%). Most (81%) pointed towards the need for further research. Only 8 out of 90 pilot studies identified by the earlier review led to subsequent main studies. Twelve studies which were interventional pilot/feasibility studies and which included testing of some component of the research process were identified through the UKCRN Portfolio database. There was no clear distinction in use of the terms 'pilot' and 'feasibility'. Five journal editors replied to our entreaty. In general they were loathe to publish studies described as 'pilot'. CONCLUSION: Pilot studies are still poorly reported, with inappropriate emphasis on hypothesis-testing. Authors should be aware of the different requirements of pilot studies, feasibility studies and main studies and report them appropriately. Authors should be explicit as to the purpose of a pilot study. The definitions of feasibility and pilot studies vary and we make proposals here to clarify terminology." [Accessed October 25, 2010]. Available at: http://www.biomedcentral.com/1471-2288/10/67.
All of the material above this paragraph is licensed under a
Creative Commons Attribution 3.0 United States License. This page was written by
Steve Simon and was last modified on
2017-06-15. The material
below this paragraph links to my
old website, StATS. Although I wrote all of the material
listed below, my ex-employer, Children's Mercy Hospital, has claimed copyright
ownership of this material. The brief excerpts shown here are included under
the fair use provisions of U.S. Copyright laws.
2007
3. Stats: IRB review of a pilot study (March 26, 2007). Dear Professor Mean: I am the new chair of the IRB at a county hospital. Many of the studies we review are pilot studies with small samples. I have been trying to locate criteria for the scientific review of pilot studies, but have not found a consensus in the literature that I have seen. Is a pilot study merely a "dry run" of the procedures that will be used in a later, larger-scale study? Or, is it reasonable for the IRB to demand that the investigator provide specific criteria for determining whether the pilot has been a success? And, should the IRB furthermore demand that specific hypotheses be formulated? My impression is that many investigators declare their studies to be pilots in order to avoid more rigorous scrutiny of their proposals.
2004
2. Stats: Design and analysis of pilot studies (September 17, 2004). A colleague sent me a very nice paper, Design and analysis of pilot studies: recommendations for good practice. G. A. Lancaster, S. Dodd, P. R. Williamson. J Eval Clin Pract 2004: 10(2); 307-12, that covers some of the same ideas in my web page, Stats: Designing a pilot study.
1999
1. Stats: Designing a pilot study (September 13, 1999). Dear Professor Mean, I am proposing a research study that will examine a complex intervention of diet, exercise, and behavioral modification for some of my pediatric patients who need to lose weight. I want to collect some data from a pilot study before I start the research study. How do I describe the pilot study in my protocol? -- Sophisticated Sarah
What now?
Browse other categories at this site