Quiz about p-values (created 2010-04-14).

This page is moving to a new website.

In one of my webinars, I offered the following quiz question:

*A research paper computes a p-value of 0.45. How would you interpret this
p-value?*

*Strong evidence for the null hypothesis**Strong evidence for the alternative hypothesis**Little or no evidence for the null hypothesis**Little or no evidence for the alternative hypothesis**More than one answer above is correct.**I do not know the answer.*

This is actually a bit of a trick question. It wasn't intended as a trick question; I was asleep at the switch when I wrote this. Still, it's like when my seven year old son makes a mistake--this really represents a learning opportunity.

Recall that a p-value is
a measure of evidence AGAINST the null hypothesis. **If the p-value is small**
(typically we say that 0.05 or less is small) then you have **lots of
evidence against the null hypothesis** and you should reject it.

**This p-value is large**, so we **do not reject the null hypothesis**.
So we can rule out #2 right away. But how about #1? We sometimes say when the
**p-value is large** that we **"accept the null hypothesis."** But
purists will insist on the wording **"fail to reject the null hypothesis."**
A large p-value means **little or no evidence against the null hypothesis**,
but you should not automatically interpret it as lots of evidence for the null
hypothesis. **It may be that your sample size is so small that you have little
evidence for ANY hypothesis**. Without some sort of context, such as an a priori
sample size justification, we can't really say too much. With that sample size
justification, #1 would be a good answer. But without it, it is possible that
we can't decide between #1 and #3. If we know that the sample size is much too
small then both #3 and #4 are correct.

But really, #6 is indeed the correct answer, though I can't be too upset with people who choose #1 or #4 or even #3 (which sort of makes #5 an attractive choice). Without more context, you cannot say anything with assurance about a large p-value. If I had provided more information, you could have chosen a specific answer, perhaps, but I didn't give you that extra information.

This is an example of how a p-value seems backwards. It is always a measure against the null hypothesis, but never, by itself, a measure for the null hypothesis. Here's how I'll word it in future webinars.

*A research paper computes a p-value of 0.45. How would you interpret this
p-value?*

*Strong evidence for the null hypothesis**Strong evidence for the alternative hypothesis**Strong evidence against the null hypothesis**Strong evidence against the alternative hypothesis**A large p-value, by itself, does not provide strong evidence for or against any hypothesis.**I do not know the answer.*