P-value

The most common misreading treats a p-value of 0.03 as a 3 percent chance the null hypothesis is correct. It says nothing of the kind. It says that if there were truly no effect, data this extreme or more so would occur 3 percent of the time. The distinction matters because a small p-value from a large sample can accompany a trivially small effect, while a large p-value can reflect low power rather than a genuine null.

The 0.05 threshold is a convention, not a law of nature, and the 2016 American Statistical Association statement warned explicitly against mechanical significance testing. Good manuscripts report effect sizes and confidence intervals alongside p-values, so readers can judge magnitude and precision rather than a binary verdict of significant or not.

In review, watch for dichotomania, the habit of treating p just under 0.05 as real and p just over as nothing, and for p-values reported without the corresponding effect estimates. A result described only as significant, with no number attached to how large the effect is, is incomplete.