2552 issues across 117 AI pre‑reviews show rigor problems outweigh writing concerns

Across 117 manuscripts, ManuscriptMind flagged 2552 issues, with methodology alone accounting for 682. The most important signal is that the tool is finding more serious problems in how studies are designed and analyzed than in how they are written.

Where are manuscripts most at risk?

The severity mix points to a clear hierarchy of risk. Methodology issues are not only the largest category, they also carry a high proportion of critical and major flags. That pattern is what you would expect if the core design choices in many manuscripts are only partially aligned with field standards: unclear inclusion criteria, weak control conditions, insufficient power, or mismatched measures.

Statistics issues form the next major block of substantive concerns. Here, a substantial share of flags are major rather than minor, which is consistent with problems like inappropriate models, incomplete reporting of effect sizes or confidence intervals, or misinterpretation of p‑values. These are the kinds of issues that can change the answer to "what did the study actually show" rather than just how clearly it is stated.

By contrast, writing and data presentation skew toward minor severity. That does not mean prose and figures are polished, but it does suggest that most detected problems in these areas are fixable without re‑running analyses or revising the study design. Typical examples would be ambiguous phrasing, dense paragraphs, unclear axis labels, or inconsistent terminology. They matter for readers, yet they rarely threaten the validity of the findings.

Conclusions and literature sit between these poles. Many conclusion issues are major, reflecting overstatement of implications, causal language that is not supported by the design, or generalizations beyond the sample. Literature problems often involve gaps in citing relevant work or over‑reliance on a narrow set of sources. These categories connect directly to how the manuscript positions its contribution, so they are less cosmetic than they may appear.

What do mid‑range rubric scores actually say?

The rubric scores for rigor, significance, and presentation cluster in the mid‑range, with significance the highest at 3.65 and rigor the lowest at 3.42 on the 1–5 scale. This pattern suggests that most manuscripts are conceptually interesting and potentially valuable, but their execution leaves room for improvement.

A mid‑range rigor score typically reflects partial adherence to best practices rather than outright methodological failure. Core elements may be present but underdeveloped: sample sizes that are adequate but not robust, measurement tools that are reasonable but not well validated, or analytic plans that address main questions but omit sensitivity checks. In many disciplines, this is exactly where peer reviewers focus their most detailed comments.

The slightly higher significance score implies that authors are often asking questions that matter to their fields and framing them in a way that reviewers recognize as relevant. The challenge is converting that conceptual promise into a design and analysis that can support strong claims.

Presentation in the mid‑range indicates that manuscripts are generally readable and structured, yet still burdened by issues that slow down comprehension. These might include uneven use of headings, dense methods sections, or figures that require effort to decode. None of these are fatal, but they can obscure the strengths of the work.

How should authors read the severity mix?

The concentration of critical and major issues in methodology, statistics, and conclusions means that the highest pre‑submission risk lies in the logic of the study rather than its surface form. A manuscript can be well written and nicely formatted yet still receive serious flags if its design does not match its claims.

Authors using AI pre‑review should therefore treat cosmetic feedback as a second pass. The first pass should focus on whether the design really answers the stated question, whether the chosen analyses are appropriate and fully reported, and whether the conclusions stay within what the data can support. Literature and data presentation issues are important, but they are often easier to correct once the core structure is sound.

What to do before your next submission

Before sending a manuscript to a journal, it is worth running through the same hierarchy that emerges from these reviews:

• Start with methodology. Check inclusion and exclusion criteria, control conditions, measurement validity, and whether the design truly maps to the research questions.
• Examine the statistical choices. Confirm that models, assumptions, and reported metrics align with current norms in your field.
• Re‑read the conclusions against the results. Remove causal or general claims that the design cannot support.
• Only then refine writing, figures, and tables, ensuring that they accurately and clearly represent the underlying analyses.

The snapshot suggests that strengthening rigor and analytic transparency will do more to reduce critical issues than polishing prose alone. Treating these as the primary targets of revision can make AI pre‑review, and later human peer review, more about sharpening contributions than repairing foundational problems.