StATS: A small p-value does not mean a large difference (February 8, 2005)
Someone asked me if the p-value for a t-test indicates the size of the difference between two groups. It turns out that the p-value is related both to the size of the difference and the sample size. In general, a very small p-value might indicate a large difference, a large sample size, or both. It would be better to use a confidence interval rather than a p-value because you won't have any ambiguity with the confidence interval.
This page was written by Steve Simon while working at Children's Mercy Hospital. Although I do not hold the copyright for this material, I am reproducing it here as a service, as it is no longer available on the Children's Mercy Hospital website. Need more information? I have a page with general help resources. You can also browse for pages similar to this one at Category: Pvalues.