Do multiple time points require a Bonferroni adjustment? (created 2009-07-18)

This page is moving to a new website.

I'm a little confused as to when to apply the multiple comparisons correction. If I had a measure which compared blood pressure (say) between two groups after 7, 14 and 21 days post procedure, would I need to adjust for multiple comparisons of the order three?

There's no consensus in the research community about when to apply the Bonferroni correction. If you will claim that a drug is efficacious if it reduces blood pressure at any of the three time points, then FDA would almost certainly require you to use a Bonferroni adjustment.

If the 21 day value is the most critical one and the 7 and 14 day values are just there because you're curious about the short term behavior, then you can designate the 21 day value as the primary endpoint and the 7 and 14 day values as secondary endpoints. In this situation, no Bonferroni adjustment is needed.

If you can only declare efficacy if you have statistical significance at all three time points, then your test needs to be adjusted in the opposite direction.

My philosophy is to do what you darn well please, but be ready to use a different approach when the referees raised an objection. I used to be more rigid about the need for a Bonferroni correction, but then I realized that most of the people I worked with understood quite well the limitations associated with testing multiple groups, multiple time points, multiple dependent variables. etc. They would discount the results appropriately, either by using a Bonferroni correction themselves or by viewing the failure to adjust for multiple comparisons as an appropriate limitation on the research, even if the authors themselves try to dodge the issue.