Reproducibility vs replicability

The terms are often confused, and definitions vary by field, but the useful distinction is between rerunning an analysis and rerunning a study. A result that cannot even be reproduced from its own data and code points to computational error or undisclosed steps. A result that reproduces but fails to replicate in new samples suggests the original finding was a fluke, overfit, or context-dependent.

Large-scale efforts revealed the scale of the problem. The Open Science Collaboration replicated fewer than half of 100 psychology studies, and comparable failures surfaced in cancer biology, economics, and beyond. Contributing causes include underpowered designs, p-hacking, HARKing, and publication bias, which together let fragile findings dominate the literature.

Remedies center on transparency: sharing data and code, preregistering analyses, powering studies adequately, and rewarding replication rather than only novelty. A manuscript that provides its data, code, and enough methodological detail for others to reproduce it is far more trustworthy, and reviewers increasingly check for exactly that.