|
|
P.Mean >>
Category >> Post hoc power (created 2007-09-11).
|
Post hoc power represents a calculation of power after the data have been collected. These pages explain why this calculation is not appropriate. Also see Category: Sample size justification, Category: Writing research papers. Other entries about post hoc power can be found in the post hoc power page at the StATS website.
2012
3. P.Mean: Post hoc power persists becauses peer-reviewers demand it (created 2012-01-04). I was in the middle of writing a grant looking at best research practices and wanted to give an example of when best practices weren't being followed. The easiest example to find was the use of post hoc power calculations. There's been at least two decades of criticism of this practice and yet it still occurs. The example I found, however, has an interesting twist to the tale.
The abuse of power: the pervasive fallacy of power calculations for data analysis. John M Hoenig, Dennis M Heisey. The American Statistician 2001: 55(1); 19-24. Description: This article demonstrates that several different approaches for calculating post-hoc power are flawed and can produce misleading conclusions. Once a confidence interval has been computed, there is no additional information that a post hoc power calculation can provide.
Smith AH, Bates MN (1992). Confidence limit analyses should replace power calculations in the interpretation of epidemiologic studies. Epidemiology; 3(5): 449-52. Abstract: "Frequently, after an epidemiologic study is completed, statistical power to detect a relative risk of interest is recalculated using data obtained during the course of the study. A negative study may then be dismissed on the grounds that its power was too low. However, post hoc power calculations ignore the actual relative estimate and its variance, which are by then known. We present evidence that post-study power calculations have little value and should be replaced by a more informative method using the upper (1 - alpha)% confidence limit of the point estimate that touches the value of the relative risk of interest."
The use of predicted confidence intervals when planning experiments and the misuse of power when interpreting results. Goodman S Annals of Internal Medicine 1994; 121(3): 200-206. [Medline] [Abstract] [Full text]. Description: An early article written for doctors that explains why you should not calculate power after the experiment is completed. These calculations have, according to the authors, an "Alice-in-Wonderland feel" because they are guaranteed to confuse the issue.
All of the material above this paragraph is licensed under a
Creative Commons Attribution 3.0 United States License. This page was written by
Steve Simon and was last modified on
2017-06-15. The material
below this paragraph links to my
old website, StATS. Although I wrote all of the material
listed below, my ex-employer, Children's Mercy Hospital, has claimed copyright
ownership of this material. The brief excerpts shown here are included under
the fair use provisions of U.S. Copyright laws.
2005
2. Stats: Post hoc power is never justified (May 13, 2005). Someone wrote in and was upset that a referee was insisting on post hoc power for all the outcome measures, and he only wanted to compute post hoc power for the negative outcomes (the outcomes that did not achieve statistical significance).
2002
1. Stats: Post hoc power (November 1, 2002). Dear Professor Mean, The results of my study were negative, and the journal reviewer insists that I perform a post hoc power calculation. How do I do this? -Jittery Jerry
What now?
Browse other categories at this site