P.Mean >> Category >> Resampling methods (created 2008-07-18). 

These pages cover resampling methods and other closely related topics like the bootsrap. Resampling is a very simple approach that offers solutions to many problems with very few assumptions about the data. These methods require a lot of computing power and have become increasingly prominent as computers have become faster. Also see Category: Adverse events in clinical trials, Category: Bayesian statistics. There is no other directly comparable page on the StATS website, but some of the topics listed under the unusual data page are relevant.

2008

  1. P.Mean: Software for bootstrap and resampling (created 2008-07-18). Someone asked me what software they should use as they were learning how to use the bootstrap and resampling methods.

Outside resources:

  1. P. Adam Kelly. Overview of Computer-Intensive Statistics. Excerpt: "Resampling procedures, also commonly referred to as computer intensive statistical inference procedures, may be used to assess the significance of a statistic in a hypothesis test or to determine the lower and upper bounds for a confidence interval when the usual assumptions of parametric statistical procedures are not met (Manly, 1991). Computer intensive procedures require the recomputation of hundreds or thousands of artificially constructed data sets. Like other nonparametric statistical procedures, these procedures existed as theory on paper long before they were brought into the practical mainstream. The Monte Carlo method of resampling, for example, was introduced by Barnard in 1963 (Noreen, 1989), but at that time could only be illustrated and implemented operationally on very small sample sizes. However, with the advent of fast, inexpensive computing, essentially since around 1990, the use of computer intensive procedures has grown dramatically, particularly in the area of basic academic research. Actually, with the widespread availability of powerful personal computers and statistical software that even brings resampling-type methods right into the home, the name computer intensive seems today to be as anachronistic as it was descriptive just a few years ago." [Accessed March 4, 2009]. Available at: http://www.hsrd.houston.med.va.gov/AdamKelly/resampling.html.
  2. David C. Howell. Resampling Statistics: Randomization and the Bootstrap. Excerpt: "This set of pages is intended to serve two purposes. On the one hand, it was written to accompany a set of Windows© programs that I have written. The main program is named Resampling.exe, and is available on disk and can be downloaded from www.uvm.edu/~dhowell/StatPages/Resampling/ResamplingPackage .zip. The second purpose of these pages is to elaborate on resampling techniques and the theory behind them. " [Accessed December 5, 2009]. Available at: http://www.uvm.edu/~dhowell/StatPages/Resampling/Resampling.html.
  3. Julian L. Simon. Resampling: The New Statistics. Excerpt: "This text grew out of chapters in the 1969 edition of Basic Research Methods in Social Science by the same author, and contains the first published example of what was later called the bootstrap. Simon is best known for his research in demography, population and the economics of natural resources, and gained fame when the noted biologist Paul Ehrlich selected five commodities and bet Simon that scarcity would drive their prices up over the period of the bet (in fact, their prices all dropped). Resampling: The New Statistics contains a number of examples in Resampling Stats, a computer program originated by Simon, but can be read on its own without the program." [Accessed December 5, 2009]. Available at: http://www.resample.com/content/text/index.shtml.

Creative Commons License All of the material above this paragraph is licensed under a Creative Commons Attribution 3.0 United States License. This page was written by Steve Simon and was last modified on 2010-04-12. The material below this paragraph links to my old website, StATS. Although I wrote all of the material listed below, my ex-employer, Children's Mercy Hospital, has claimed copyright ownership of this material. The brief excerpts shown here are included under the fair use provisions of U.S. Copyright laws.

2008

P.Mean: Eliciting a prior distribution for rejection/refusal rates (June 7, 2008). I got a question about the Bayesian model for rejection/refusal rates. I had used three prior distributions in my calculations, a Beta(10,40), a Beta(45,5), and a Beta(25,25). The question was, how did I select those prior distributions.

P.Mean: A simple Bayesian model for exponential accrual times (May 26, 2008). Here is a simple Bayesian model for exponential accrual times. This model will help researchers to plan the estimated duration of a clinical trial. The same model will also allow the researcher to monitor the accrual during the trial itself and develop revised estimates for the duration or the sample size.

P.Mean: Why does a Bayesian approach make sense for monitoring accrual? (May 8, 2008). I'm working with Byron Gajewski to develop some models for monitoring the progress of clinical trials. Too many researchers overpromise and undeliver on the planned sample size and the planned completion date of their research This leads to serious delays in the research and inadequate precision and power when the research is completed. We want to develop some tools that will let researchers plan the pattern of patient accrual in their studies. These tools will also let the researchers carefully monitor the progress of their studies and let them take action quickly if accrual rates are suffering. We've adopted a Bayesian approach for these tools. While a Bayesian approach to Statistics is controversial, we feel that there should be no controversy with regard to using Bayesian models in modeling accrual.

P.Mean: Slipped deadlines and sample size shortfalls in a random sample of research studies (May 7, 2008). There is a limited amount of data out there that suggests that many researchers overpromise on the planned sample size and completion date and underdeliver. About a year ago, I received a small grant to study the proportion of studies at Children's Mercy Hospital (CMH) that failed to meet the proposed completion deadlines, that failed to recruit the promised number of patients or both. Here is a brief summary of these results.

P.Mean: Monitoring refusals and exclusions in a clinical trial (May 1, 2008). Someone sent me an email asking about the work that Byron Gajewski and I have done on monitoring accrual patterns in clinical trials. She had been doing something similar at her job and wanted to see if we could collaborate. In her situation, the major issue was the number of patients who made an initial contact but did not keep their first appointment, the number of patients who kept the appointment, but refused to sign the consent form once they realized what the study was about, and the number of patients who did sign the consent form, but who did not meet the inclusion criteria once the initial screening was done.

2007
 

P.Mean: Case study of accrual in a clinical trial (September 11, 2007). I received additional accrual data on a clinical trial I am monitoring. To review, the trial started on August 28, 2007 and will continue until January 31, 2008, for a total of 22 weeks. The researcher thinks that he might be able to get 3 patients per week over a 22 week trial (66 total), but he is very confident that he would get at least 2 patients per week (44 total). The confidence in the estimate of 3 patients per week was rated as 5 on a 10 point scale. After one week, a single patient has entered the study. No patients enter on weeks 2, 3, or 4. On week 5, three patients enter the study. On week 6, one more patient enters for a total of 5 patients.

P.Mean: An alternate way of viewing accrual (October 2, 2007). I was talking about a project with a fellow in Emergency Medicine and during the discussion realized a different way of looking at accrual in a clinical trial. She plans to look how accurately EKGs are read by physicians in the Emergency Room. I showed her some of the work that Byron Gajewski and I had done on planning and monitoring accrual rates. She pointed at that accrual was not a problem here in that the number of EKGs that are processed in the ER is known with very high precision. The problem, of course, is that the physicians who participate in the study have to fill out a small amount of additional paperwork for the research. While this is not an intrusive amount of work and she is going to work hard to promote this research project, there will some physicians at some times who will not fill out the extra research paperwork, or will fill it out so incompletely as to make the EKG unusable in the research. The ER is a busy and hectic place and it is difficult to get complete data, even when the ER doctors are trying their best to help with the research.

P.Mean: Case study of accrual in a clinical trial (September 11, 2007). Someone came by today with a project where he wants to monitor the accrual in a clinical trial. The trial started on August 28, 2007 and will continue until January 31, 2008, for a total of 22 weeks. He thinks that he might be able to get 3 patients per week over a 22 week trial (66 total), but he is very confident that he would get at least 2 patients per week (44 total).

P.Mean: Accrual grant, Round 3 (August 21, 2007). Last year, I applied for a Kansas City Area Life Sciences Institute (KCALSI) Research Development grant. It was not funded, but a subsequent grant that I submitted to the Katherine B. Richardson foundation was funded. Both grants are rather small, intended as seed money to encourage development of a larger scale project which might attract funding from the NIH or a large foundation. I want to revise the KCALSI grant and re-submit it for the 2007 cycle.

2006
 

P.Mean: A simple Bayesian model for accrual (November 17, 2006). Suppose you are a researcher in charge of a long term study. You plan to collect data on 120 patients. The goal is to finish your study in ten years, which means getting 12 patients per year or one every thirty days on average. Recruiting patients though appears to be harder than you had expected. You recruited your first patient on day 56, 26 days behind schedule. The second patient is not recruited until day 93. About two years into the study (day 768), you have just recruited your 10th patient. It looks like recruitment might be behind schedule. Is it time to take action? A Bayesian model of accrual times can help you to discern whether recruitment is behind schedule and project an estimated completion date allowing for uncertainty.

P.Mean: My second grant, part 3 (October 2, 2006). I just finished my second grant, which I gave the title "Estimating delays in completion of IRB approved and KBR supported research studies" The two acronyms, IRB and KBR should be familiar to the group I am applying to. IRB stands for Institutional Review Board and KBR represents an internal grant mechanism here at Children's Mercy Hospital to support initial research efforts. The initials KBR stand for Katherine Berry Richardson, who is one of the initial founders in Children's Mercy Hospital.

P.Mean: My second grant, part 2 (September 13, 2006). I took a three day workshop on grant writing and prepared a draft grant as part of the student exercises in that class. It's not in the format that I need to use, but it outlines most of the goals and efforts of my proposed work. I wrote about accrual problems in clinical trials.

P.Mean: My second grant (July 26, 2006). I'm in the final stretch of writing a grant to submit to the Kansas City Area Life Sciences Institute. I am already thinking "what is my next step?" One possibility would be to run a small study that will provide hard numbers to support a commonly expressed belief that most research studies fall behind schedule and fail to get anything close to the targeted sample sizes.

P.Mean: Initial work on the KCALSI grant (July 17, 2006). I am submitting a grant in response to a KCALSI RFP. According to the RFP, the general structure of the grant should follow the structure used by NIH. Here is a review of the structure of a typical NIH grant.

P.Mean: Early detection of accrual problems in clinical trials (June 30, 2006). The most common reason why clinical trials fail is that they fall well below their goals for patient accrual. Institutional Review Boards (IRBs) are charged with the continual monitoring of clinical trials and they need to identify when these trials encounter problems with accrual. When do they "jump the shark" so to speak?

P.Mean: Applications of the CUSUM chart (June 20, 2006). I am interested in investigating the use of CUSUM charts in monitoring accrual rates, drop out rates, and adverse event rates in a clinical trial. Some references which I might cite in a literature review are listed here.

P.Mean: Monitoring accrual rates (May 30, 2006). This scenario is based on real data, but has been adapted slightly to serve as an illustration of the use of control charts in monitoring a clinical trial. Suppose a clinical trial was set up in 1997 and the goal was to recruit one patient per month over a ten year period, for a total sample size of 120 patients. Here are the dates of recruitment for the first 42 patients.

What now?

Browse other categories at this site

Browse through the most recent entries

Get help