ARD-Review:
Generating Accurate, Reasonable, and Diverse AI Peer Review

Better review does not mean more criticism

AI review systems are becoming useful pre-submission tools, but a familiar failure mode remains: they often produce feedback that sounds plausible without being important, specific, or even valid for the paper at hand. AI reviewers often repeat the same generic checklist, raise issues outside a paper's intended scope, or turn marginal concerns into decisive weaknesses. We group these failure modes as GUNI [1]:

The problem compounds when many papers receive the same advice. Authors respond by adding more ablations, qualifications, citations, and defensive explanations. The review becomes longer, the paper becomes longer, and neither necessarily becomes better.

It also pushes very different papers toward the same shape: the same standard ablations, the same caveat-heavy framing, the same checklist-driven appendix, and the same flattened academic voice. Authors begin optimizing for a recurring review template instead of presenting each contribution in the form that best fits it. By filtering out generic and out-of-scope requests, ARD-Review aims to reduce this homogenizing pressure while preserving criticism that is specific to the paper.

ARD-Review starts from a different objective: find the issues that matter, remove the ones that do not, and assign a score relative to real reviewing standards.

Two agents, three jobs

The pipeline, shown in Figure 1, deliberately gives different stages different responsibilities.

This changes the scoring question from "What number feels right?" to "How does this paper compare with papers that human reviewers scored before?"

A benchmark for accuracy, validity, and diversity

We introduce ARD-Bench using ICLR 2026 papers and their pre-rebuttal reviewer scores. Pre-rebuttal scores preserve independent reviewer judgments and align with the setting in which an AI reviewer reads a paper without seeing author responses or other reviews. The evaluation covers three dimensions:

• Scoring accuracy: Spearman and Pearson correlation, mean absolute error (MAE), bias, and AUROC against human reviewer scores.
• Review validity: pairwise judgments of which review contains more accurate, substantiated, constructive, and professional feedback, informed by the deficiency dimensions studied in TADDLE [6].
• Review diversity: cross-paper overlap between the weaknesses raised for different papers, following the motivation of prior work on review homogenization [5]. Lower overlap means the reviews are more paper-specific.

The results

The CSPaper-hosted version of ARD-Review uses GLM-5.3-Flash [7]. It reaches 0.5400 Spearman, 0.5868 Pearson, and 0.7317 AUROC. The DeepSeek-V4-Flash configuration reaches 0.5487 Spearman, 0.6137 Pearson, and 1.3335 MAE. Among the evaluated non-commercial systems, the DeepSeek configuration has the strongest overall scoring accuracy, and its scoring accuracy is statistically on par with CSPaper-Offline [4]. ARD-Review is also the only non-commercial method that significantly outperforms the human one-vs-one baseline across Spearman correlation, Pearson correlation, and MAE.

Scoring is only one part of a useful review. In the pairwise validity evaluation, ARD-Review wins 85.53% of all comparisons (Figure 3).

Additionally, aggressive filtering does not collapse the output into a generic template. ARD-Review's cross-paper overlap is 0.508664, close to Direct Review. The system removes weak comments while preserving paper-specific criticism.

The calibration step travels well

The same pipeline works across different model backbones. With Claude Sonnet 4.6 [8], ARD-Review reaches 0.6376 Spearman, 0.7000 Pearson, and 0.7879 AUROC, significantly exceeding the human one-vs-rest correlation baseline reported in the paper. A 27B Qwen model [9] also remains competitive with human baselines, making local review possible on a single high-end GPU.

Calibration also enables two practical outputs beyond the review score. Given an AI-predicted score s, ARD-Review can report the empirical acceptance chance P(accept | AI score = s) and the score's percentile among papers scored by the same reviewer configuration. The probability is conditioned directly on the AI's predicted score, not on a human score inferred from it, so there is no accumulated error from a second score-mapping step.

Try ARD-Review

ARD-Review is now one of the review options available through CSPaper, so the method can be tested without setting up the pipeline locally. The CSPaper-hosted option uses GLM-5.3-Flash. The Claude Sonnet 4.6 configuration is available only through the GitHub repository, because Claude Agent SDK access through a subscription is for personal use only. The paper is available on ResearchGate [1], and an arXiv link is coming.

A useful AI reviewer should not merely produce more comments. It should identify the comments that deserve an author's attention, explain them clearly, and score the paper against a real standard.

Host your method on CSPaper

ARD-Review is the first method agent, and we do not intend it to be the last. Method agents are a new agent category built for external contributions: if you have developed a peer-review method, backed by a paper or an open-source implementation, CSPaper can host it as a review option under your own name, with links to your code and paper. You keep the credit; our users get to run your method without setting anything up.

If you would like to see your method hosted on CSPaper, contact us at support@cspaper.org. Tell us briefly what the method does and where the paper or repository lives, and we will follow up.

Acknowledgements

This article was drafted by Wenqi Marshall Guo, the first author of ARD-Review, and edited and published by the CSPaper team. ARD-Review was developed by Wenqi Marshall Guo, Shiyu Zhou, Guangzeng Han, Qingyun Qian, Mohamed Shehata, and Shan Du [1]. The code is open source at github.com/weathon/ARD-Review, and the paper is available on ResearchGate at doi.org/10.13140/RG.2.2.34444.48005. An arXiv link is coming.

References