Reading Statistical Significance Without Misleading Yourself
A small p-value does not mean a result is large or important. Learn what significance tells you, and what effect sizes and confidence intervals add.

"Statistically significant" is one of the most misread phrases in research. It sounds like a verdict on importance, but it is a narrower statement about how surprising your data would be if there were no real effect.
What a p-value is
A p-value is the probability of seeing data at least as extreme as yours, assuming the null hypothesis is true. A p-value of 0.03 does not mean there is a 97 percent chance the effect is real.
Why significance is not importance
With a very large sample, tiny and practically meaningless differences can reach p < 0.05. With a small sample, a large and important difference can fail to. Always ask how big the effect is, not only whether it passed a threshold.
Report these alongside p
- Effect size: the magnitude of the difference or relationship, in units people can interpret.
- Confidence interval: the range of effects consistent with your data. A wide interval means you know less than a single p-value suggests.
- Sample size and design: so readers can judge how much weight to give the result.
Common traps
- Running many tests and reporting only the significant ones.
- Treating p = 0.049 and p = 0.051 as completely different findings.
- Reading "not significant" as "no effect".
Significance answers "could this be chance?" It does not answer "does this matter?".
Plan your tests in advance, report the full set, and describe results in terms of size and uncertainty. Your readers will thank you.
