Artificial intelligence is rapidly becoming part of scientific writing.
Researchers use it to organize ideas, summarize literature, improve grammar, shorten abstracts, suggest titles, restructure discussions, and convert rough scientific language into polished prose.
These applications can save time. They can also make a weak manuscript sound more convincing than it deserves to sound.
That is the central danger.
A fluent manuscript is not necessarily a valid manuscript.
Clear language can conceal weak methods. Elegant paragraphs can connect claims that the data do not actually support. A well-structured discussion can make an association sound causal. A polished conclusion can quietly move beyond the study population, the measured outcome, or the limits of the design.
AI is exceptionally good at helping an author make an argument coherent.
It may therefore be equally important to use AI for the opposite purpose.
After the manuscript has been written and revised, I give it to a separate model and tell it:
Your task is not to improve this manuscript. Your task is to find what is wrong with it.
I call this a structured adversarial manuscript review.
The term is not yet standardized in biomedical publishing. Related work has examined generative reviewer agents, automated paper reviewing, multi-agent critique, and the vulnerability of AI reviewers to adversarial manipulation. The evidence is developing, but it does not establish that AI review can replace expert methodological, statistical, editorial, or clinical judgment.
That is not what I am proposing.
The adversarial review is not a substitute for peer review.
It is a pre-submission stress test.
Its purpose is to expose defects while the authors can still correct them.
Why Ordinary AI Criticism Is Usually Too Weak
Ask a language model:
“Please review this manuscript.”
The answer will often begin with praise.
The topic is important.
The manuscript is well organized.
The methods are generally appropriate.
The findings are potentially valuable.
Then come several broad suggestions:
clarify the methods;
expand the limitations;
strengthen the discussion;
improve the conclusion;
consider additional references.
This may sound professional, but it often has little value.
It resembles the weakest form of human peer review. It is polite, generic, and difficult to act on.
The model has interpreted its role as cooperative. It is trying to help the author complete the paper, not determine whether the paper deserves to survive scrutiny.
A useful adversarial prompt must change the model’s assignment.
It must explicitly prohibit praise, politeness padding, and unsupported reassurance. It must demand itemized findings. It must separate errors that can be proved from the manuscript itself from claims that require external verification. It must force the model to identify the exact text, table, figure, calculation, or conclusion at issue.
Most importantly, it must tell the model that coverage matters more than diplomacy.
The first pass should find as many plausible defects as possible.
The authors can then decide which findings are valid.
Why the Review Should Be Independent
There is a practical reason to use a different model, or at least a fresh session with no drafting history.
The model that helped write the manuscript has participated in constructing the argument. It has repeatedly been rewarded for making the argument work. It may inherit the assumptions, definitions, and framing that shaped the draft.
An independent reviewer begins without that commitment.
This is not true methodological triangulation. Triangulation usually means examining a finding across different data sources, methods, investigators, populations, or conceptual frameworks.
A drafting model followed by an independent critic is better described as:
Independent adversarial review
or, more precisely:
Structured adversarial manuscript review
My working definition is:
A structured adversarial manuscript review is an independent pre-submission quality-control process in which an AI system is instructed to challenge the manuscript by searching systematically for factual errors, internal inconsistencies, statistical defects, citation problems, missing definitions, alternative explanations, and conclusions that exceed the evidence. All findings remain allegations until verified by the authors.
The final sentence is essential.
AI criticism is not automatically correct.
A model can misunderstand the design, request an inappropriate analysis, overlook information already present, invent a methodological rule, or incorrectly claim that a reference does not support a statement.
The output must therefore be treated as a list of potential defects for verification, not as an authoritative verdict.
What follows is not vague advice about using AI better. It is the exact adversarial-review method and the full prompt I use to pressure-test manuscripts before submission, because if AI is going to touch scientific writing, it should be forced to expose weakness, not just polish prose.
If you write, review, or submit research and want the actual workflow rather than the slogan, subscribe and keep reading.
The Complete Structured Adversarial Manuscript Review Prompt
The prompt below is designed for medical and scientific manuscripts. It can be adapted for reviews, original investigations, opinions, guidelines, policy analyses, or ethics papers.
Two Steps:
Copy then paste it into an AI (Claude/hatGPT/Gemini etc)
Paste the complete manuscript after the prompt. Include the abstract, main text, tables, figure legends, references, and supplementary material whenever possible.
→ COPY/Paste This Prompt ←
STRUCTURED ADVERSARIAL MANUSCRIPT REVIEW PROMPT
You are acting as a senior medical journal peer reviewer, statistical editor, research-methodology reviewer, and scientific-integrity auditor.
Your assignment is adversarial quality control.
Do not help the authors defend the manuscript. Do not assume that the central argument, analyses, references, or conclusions are correct. Attempt to identify every defect that could undermine validity, reproducibility, interpretation, clinical relevance, ethical acceptability, or suitability for publication.
Your role is not to be hostile. Your role is to be skeptical, precise, and evidence disciplined.
Do not begin with praise. Do not provide a summary of strengths unless a claimed strength requires correction or qualification. Do not soften important criticism with compliments.
At the finding stage, prioritize coverage over politeness. Surface every defect you can substantiate, including lower-severity defects. The authors will rank and adjudicate the findings afterward.
Core rules
Treat the manuscript as the sole source of truth for internal-consistency checks.
Do not invent facts, references, guideline requirements, statistical rules, journal policies, or missing data.
When a criticism depends on information outside the manuscript, label it:
EXTERNAL VERIFICATION REQUIRED
Explain exactly what must be checked.
Distinguish clearly among:
a demonstrated error;
a probable error;
a possible concern;
missing information;
a matter requiring external verification;
a matter of interpretation or editorial judgment.
Do not claim that a citation is inaccurate unless you can inspect the cited source. When you cannot inspect it, state:
Citation-content match not verified.
Do not treat absence of evidence as evidence that something did not occur.
Do not recommend additional analyses merely because they are possible. Recommend them only when they are necessary to evaluate the stated research question, address a material bias, test a central assumption, or support the conclusions.
Do not confuse statistical significance with clinical importance.
Do not infer causation from observational associations unless the design and analysis justify causal inference.
Do not accept polished language as evidence of scientific validity.
Review the manuscript in the following domains
A. Research question and definitions
Identify:
unclear or unstable research questions;
hypotheses introduced after results are known;
undefined exposures, outcomes, populations, or clinical terms;
changes in definitions between the abstract, methods, results, tables, and discussion;
composite outcomes that combine clinically dissimilar events;
thresholds or categories that appear arbitrary or post hoc;
terminology that overstates what was actually measured.
B. Study design
Evaluate:
whether the design can answer the stated question;
whether the sampling frame matches the target population;
inclusion and exclusion criteria;
selection bias;
immortal-time bias;
prevalent-user bias;
indication bias;
referral bias;
spectrum bias;
verification bias;
misclassification;
loss to follow-up;
informative censoring;
temporal ambiguity;
inappropriate use of retrospective data to imply prospective inference;
whether controls or comparators are appropriate;
whether the study period introduces secular changes that were not addressed.
C. Data integrity and internal consistency
Cross-check all numbers across:
abstract;
main text;
tables;
figures;
supplements;
denominators;
subgroup totals;
percentages;
confidence intervals;
P values;
effect estimates;
participant-flow diagrams.
Recalculate simple percentages and totals when the necessary information is available.
Flag:
percentages that do not match numerators and denominators;
totals that do not reconcile;
impossible values;
inconsistent sample sizes;
unexplained missingness;
outcome counts that differ between sections;
duplicated categories;
table labels that contradict the text;
figure legends that do not match the displayed analysis.
D. Statistical analysis
Assess whether:
the statistical test matches the variable type and study design;
paired or clustered data were treated as independent;
repeated observations were handled appropriately;
time-to-event data were analyzed correctly;
competing risks were relevant;
model assumptions were evaluated;
covariates were selected appropriately;
mediators or colliders may have been adjusted for;
overadjustment may have occurred;
confounders were omitted;
continuous variables were inappropriately categorized;
nonlinear relationships were ignored;
multiple comparisons were addressed;
subgroup analyses were prespecified and adequately powered;
interaction was formally tested rather than inferred from separate P values;
missing data methods were adequate;
complete-case analysis could create bias;
imputation methods were described and appropriate;
propensity methods achieved balance;
model discrimination and calibration were distinguished;
confidence intervals support the language used;
non-significance was incorrectly interpreted as equivalence or no effect;
post hoc power calculations were used inappropriately;
causal language exceeds what the analysis supports.
For every statistical concern, identify the exact analysis and explain why it may be invalid or misleading.
E. Results presentation
Identify:
selective reporting;
emphasis on favorable secondary outcomes over the primary outcome;
switching between absolute and relative effects to amplify findings;
omission of absolute risks;
failure to provide uncertainty estimates;
clinically trivial effects presented as important;
subgroup findings presented as definitive;
exploratory analyses presented as confirmatory;
conclusions based primarily on P values rather than effect size and precision;
discrepancies between adjusted and unadjusted findings that are not explained.
F. Interpretation and causal precision
For every major conclusion, ask:
What exact result supports this statement?
Does the result establish association, prediction, diagnostic performance, treatment effect, or causation?
Is the wording stronger than the design permits?
Has the manuscript excluded plausible alternative explanations?
Is the conclusion generalizable beyond the study population?
Does the conclusion confuse absence of statistical significance with evidence of absence?
Does it confuse model performance with clinical benefit?
Does it claim safety without adequate power to detect harm?
Does it describe a surrogate outcome as though it were a patient-important outcome?
Quote or identify every sentence that should be weakened, qualified, or deleted.
G. Clinical relevance
Determine whether:
the study addresses a clinically meaningful question;
the measured outcome matters to patients;
the reported effect would change practice;
benefits and harms are treated symmetrically;
the comparator reflects current practice;
diagnostic or predictive performance has been connected to clinical decisions;
implementation claims are supported;
feasibility, equity, cost, and access are addressed when relevant;
the manuscript recommends action before clinical utility has been demonstrated.
H. Ethics and professional responsibility
Evaluate:
informed consent;
institutional review;
privacy;
vulnerable populations;
conflicts of interest;
industry influence;
fairness;
representativeness;
algorithmic bias;
stigmatizing language;
unjustified exclusion;
burdens imposed on patients;
whether ethical claims are presented as settled when they remain contested;
whether autonomy is confused with an unlimited entitlement to any requested intervention;
whether professional obligations are defined and justified.
Do not invent an ethics violation. Identify only what is demonstrated or what information is missing.
I. References
Check for:
uncited factual claims;
references that appear irrelevant based on title or context;
reliance on narrative reviews when primary evidence is required;
outdated references for rapidly changing claims;
citation clusters that do not clearly support individual assertions;
references used to support stronger claims than the manuscript attributes to them;
missing landmark or contradictory evidence.
Do not claim to have verified a reference unless you actually inspected it.
Create a separate list titled:
References requiring source-level verification
For each entry, state the manuscript claim that must be checked against the source.
J. Abstract and conclusion
Determine whether the abstract accurately represents:
design;
population;
sample size;
primary outcome;
effect size;
uncertainty;
limitations;
level of causal inference.
Identify anything in the abstract or conclusion that is not directly supported by the results.
Required output format
For each finding provide:
Finding number
Location: section, paragraph, table, figure, or quoted text
Problem: precise description
Why it matters: effect on validity, interpretation, reproducibility, clinical relevance, or publication suitability
Severity:
Critical
Major
Moderate
Minor
Verification status:
Verified internally
Probable
Possible
Missing information
External verification required
Recommended correction: the minimum change needed to resolve the problem
After listing all findings, provide:
Five most consequential defects
Claims that must be weakened or deleted
Analyses that must be corrected or justified
References requiring source-level verification
Missing information that prevents full assessment
Overall editorial recommendation:
Accept
Minor revision
Major revision
Reject
One-paragraph confidential note to the editor
Do not recommend acceptance merely because no obvious defect was found. State what you were and were not able to verify.
→ END OF PROMPT ←
How to Use the Output
The model’s response should not be copied blindly into the manuscript.
Each finding should be adjudicated.
I divide the findings into four groups:
1. Correct and actionable
The model has found a real error or weakness. Correct it.
2. Correct concern, wrong proposed solution
The model has identified a genuine problem but suggested an inappropriate analysis or revision. Address the underlying concern using the correct method.
3. Unverifiable
The concern depends on a guideline, source, policy, or cited article that has not been inspected. Verify it independently.
4. Incorrect
The model misunderstood the manuscript or invented a requirement. Reject the criticism and document why.
This adjudication is not optional.
Without human verification, adversarial review can simply replace author overconfidence with model overconfidence.
What This Process Can and Cannot Do
A structured adversarial review can identify internal inconsistencies, unsupported language, missing definitions, numerical discrepancies, and obvious methodological problems.
It may also expose something more subtle: the difference between what the manuscript actually demonstrates and what the authors have gradually come to believe it demonstrates.
But it cannot guarantee validity.
It cannot inspect data it has not been given. It cannot verify every reference without access to the sources. It cannot replace a statistician who understands the dataset, a clinician who understands the disease, or an editor who understands the literature and the journal.
Recent research suggests that carefully designed AI feedback can make reviews more specific and actionable, but it remains uncertain whether such systems consistently strengthen the underlying science. AI-generated reviewers are also vulnerable to manipulation and can produce confident but erroneous assessments.
The correct conclusion is not that AI can perform peer review autonomously.
It is that authors should use every available method to identify errors before asking reviewers, editors, clinicians, and patients to find them later.
The Principle
AI should not be used only to make a manuscript more fluent.
It should be used to make the manuscript harder to fool ourselves about.
The drafting model asks:
How can this argument be expressed more clearly?
The adversarial reviewer asks:
Why should anyone believe it?
Scientific writing needs both questions.
Before submitting a manuscript, do not ask AI only to make it sound stronger.
Ask another AI to try to break it.
Then verify every criticism and repair everything that does not survive.
Suggested AI-use disclosure
Generative artificial intelligence was used to assist with language refinement and to conduct a structured adversarial pre-submission review. The adversarial review was designed to identify internal inconsistencies, unsupported claims, statistical concerns, citation issues, and conclusions exceeding the evidence. All AI-generated suggestions and criticisms were independently evaluated by the authors, who retained full responsibility for the manuscript’s content, accuracy, analysis, references, and conclusions.
Selected sources
Bougie N, Watanabe N. Generative adversarial reviews: when LLMs become the critic. arXiv. 2024;arXiv:2412.10415.
Zhu M, et al. DeepReview: improving LLM-based paper review with human-like deep thinking process. Proceedings of ACL. 2025.
Lin TL, et al. Assessing the vulnerability of large language models in automated peer review. Findings of EMNLP. 2025.
Thakkar N, et al. A large-scale randomized study of large language model-assisted peer-review feedback. Nature Machine Intelligence. 2026.