Peer review and methodology glossary
Plain-language definitions of the terms that decide whether a manuscript survives peer review, from statistics and study design to research integrity and publishing.
Peer Review
Desk rejection
A desk rejection is an editor's decision to reject a manuscript without sending it out for external review. It usually happens within days and reflects a mismatch with the journal's scope, insufficient novelty or significance, clear methodological flaws, or failure to meet submission requirements.
Open peer review
Open peer review is an umbrella term for models that lift the anonymity or secrecy of traditional review. It can mean revealing reviewer identities, publishing the review reports alongside the article, opening comments to the wider community, or all three. The goal is greater transparency and accountability.
Peer review
Peer review is the evaluation of scholarly work by independent experts in the same field before publication or funding. Reviewers assess validity, originality, methodology, and significance, then recommend acceptance, revision, or rejection. It is the primary quality-control mechanism of academic publishing, though it catches errors imperfectly.
Single-blind vs double-blind review
In single-blind review, reviewers know the authors' identities but authors do not know the reviewers'. In double-blind review, both sides are anonymous. Double-blinding aims to reduce bias tied to author prestige, gender, institution, or nationality, at the cost of imperfect anonymity when work is easily identifiable.
Study Design
Confounding
Confounding occurs when a third variable is associated with both the exposure and the outcome, creating a spurious or distorted association between them. A confounder offers an alternative explanation for an apparent effect, and failing to account for it is a leading reason observational studies reach wrong conclusions.
Immortal time bias
Immortal time bias is a distortion in cohort studies where a span of follow-up during which the outcome cannot occur is misattributed to a treatment group. Because members must survive event-free long enough to receive the treatment, the treated group gains a spurious survival advantage built into the design.
Intention-to-treat
Intention-to-treat analysis evaluates randomized trial participants in the groups to which they were assigned, regardless of whether they completed or even received the treatment. By preserving randomization, it guards against bias from non-random dropout and non-adherence and gives a realistic estimate of a treatment strategy's effect.
Preregistration
Preregistration is the practice of publicly documenting a study's hypotheses, methods, and analysis plan in a timestamped registry before collecting or examining the data. It creates a fixed record that separates confirmatory tests from exploratory ones and makes p-hacking and HARKing far easier to detect.
Selection bias
Selection bias arises when the people included in a study, or retained through it, differ systematically from the population the results are meant to describe. Because the sample is unrepresentative in ways related to the outcome, the findings can be distorted regardless of how large or well-analyzed the study is.
Surrogate endpoint
A surrogate endpoint is a measurable marker used to stand in for a clinical outcome that matters to patients, such as using tumor shrinkage or blood pressure instead of survival. Surrogates speed up trials, but they mislead when improving the marker does not translate into the patient benefit it is assumed to predict.
Statistics
Confidence interval
A confidence interval is a range of plausible values for an estimate, calculated so that, over many repeated studies, a stated percentage (usually 95) of such intervals would contain the true value. It communicates both the size of an effect and the precision with which it was measured.
Effect size
An effect size measures the magnitude of a relationship or difference, independent of sample size. Examples include Cohen's d, correlation coefficients, odds ratios, and mean differences. Unlike a p-value, which only signals whether an effect is detectable, an effect size tells you whether it is large enough to matter.
Multiple comparisons problem
The multiple comparisons problem is the inflation of false-positive risk that occurs when many statistical tests are run on the same data. At a 5 percent threshold, testing 20 independent hypotheses yields roughly a two-thirds chance of at least one spurious significant result unless the threshold is adjusted.
P-hacking
P-hacking is the practice of trying many analytic choices and reporting only the ones that push a result below the conventional significance threshold. By testing multiple outcomes, subgroups, or model specifications and keeping the flattering ones, researchers manufacture statistically significant findings that will not replicate.
P-value
A p-value is the probability of observing data at least as extreme as what was measured, assuming the null hypothesis is true. It quantifies how surprising the result is under a model of no effect. It is not the probability that the hypothesis is true, nor a measure of effect size.
Statistical power
Statistical power is the probability that a study will detect an effect of a given size when that effect truly exists. It depends on sample size, effect size, variability, and the significance threshold. Underpowered studies miss real effects and, paradoxically, exaggerate the ones they do find.
Reporting
CONSORT
CONSORT is the standard reporting guideline for randomized controlled trials, comprising a 25-item checklist and a participant flow diagram. It specifies what a trial report must disclose, including randomization, blinding, primary outcomes, and participant flow, so readers can judge validity and reproduce the design.
PRISMA
PRISMA is a reporting guideline for systematic reviews and meta-analyses, providing a checklist and a flow diagram that document how studies were searched, screened, included, and excluded. Following it makes a review transparent and reproducible and is required or recommended by most journals publishing evidence syntheses.
Publication bias
Publication bias is the tendency for studies with positive, significant, or novel results to be published more readily than those with null or negative findings. It skews the visible literature toward exciting effects, inflates meta-analytic estimates, and leaves genuine but unglamorous results stranded in the file drawer.
STROBE
STROBE is a reporting guideline for observational studies, covering cohort, case-control, and cross-sectional designs. Its 22-item checklist specifies what such studies should report, including how participants were selected, how variables were measured, and how confounding was addressed, so their strengths and limits can be assessed.
Research Integrity
Conflict of interest
A conflict of interest exists when financial, professional, or personal relationships could bias, or appear to bias, a researcher's judgment. Industry funding, patents, consulting fees, and personal ties are common examples. Disclosure lets editors, reviewers, and readers weigh the risk, and undisclosed conflicts erode trust in the findings.
Data fabrication vs falsification
Fabrication is inventing data or results that were never collected, while falsification is manipulating real data, materials, or procedures so the record misrepresents what actually happened. Together with plagiarism, they form the core categories of research misconduct and are grounds for retraction and severe professional sanction.
Hallucinated citation
A hallucinated citation is a reference that does not correspond to any real publication, typically produced when a large language model invents plausible-looking authors, titles, journals, and DOIs. As AI writing tools spread, these fabricated references are appearing in manuscripts, where they can pass a casual glance but fail verification.
HARKing
HARKing means presenting a hypothesis invented after seeing the data as if it had been predicted in advance. Coined by Norbert Kerr, it turns an exploratory finding into a falsely confirmatory one, overstating how well the evidence supports the claim and inflating the risk of chance results.
Reproducibility vs replicability
Reproducibility means obtaining the same results from the original data and analysis code, while replicability means obtaining consistent results from a new study collecting fresh data. Both are pillars of credible science, and widespread failures of replication across fields are known as the reproducibility crisis.
Retraction
A retraction is the formal withdrawal of a published article by the journal or authors, signaling that its findings are unreliable and should not be cited as valid. Retractions follow serious errors, misconduct such as fabrication or plagiarism, or unresolvable irreproducibility, and the notice should state the reason.
Salami slicing
Salami slicing is splitting a single coherent study into multiple thin papers to maximize publication count. Each fragment adds little, the pieces share data or samples without cross-referencing, and the practice inflates a CV while fragmenting the literature and distorting evidence for anyone trying to synthesize it.
Publishing
H-index
The h-index is an author-level metric equal to the largest number h such that the researcher has h publications each cited at least h times. Proposed by Jorge Hirsch in 2005, it attempts to capture productivity and impact in a single figure, but it favors long careers and disadvantages newer researchers.
Impact factor
The journal impact factor is the average number of citations in a given year to articles a journal published in the two preceding years. Reported by Clarivate's Journal Citation Reports, it is widely used as a proxy for journal prestige and widely criticized for being misapplied to individual papers and researchers.
ORCID
ORCID is a free, persistent digital identifier that uniquely distinguishes an individual researcher from everyone else, including others with the same or similar names. Rendered as a 16-digit number, it links a scholar to their publications, grants, and affiliations across systems, solving the problem of author name ambiguity.
Predatory journal
A predatory journal charges publication fees while providing little or no genuine peer review, editorial oversight, or archiving. These outlets exploit the pay-to-publish open-access model, spam researchers for submissions, and promise implausibly fast acceptance, undermining the credibility of the work they carry.
Preprint
A preprint is a complete scholarly manuscript posted to a public server before, or in parallel with, formal peer review. Servers such as arXiv, bioRxiv, medRxiv, and SSRN let researchers share findings immediately, establish priority, and gather feedback, at the cost that the work has not yet been vetted.
Registered report
A registered report is a publishing format in which peer review happens in two stages. Reviewers evaluate the introduction and methods before data collection, and a journal grants in-principle acceptance based on the question and design. The results are published regardless of whether they are positive, null, or mixed.