Researchers propose RubricReviewer, a rubric-driven framework to improve LLM-assisted academic peer review

Peer review at top academic venues is under unprecedented submission pressure, motivating the use of large language models (LLMs) as review assistants. Existing LLM-based reviewers typically map manuscripts directly to reviews, leaving the underlying assessment criteria implicit and entangling rubric derivation with final judgment.

Researchers introduce RubricReviewer, a fully rubric-driven framework that addresses these limitations by making rubric generation an explicit intermediate step. Both review generation and the final assessment are conditioned on paper-adaptive rubrics, aiming to improve objectivity and coverage.

The framework combines two components: Scout, a training-free agent that gathers external evidence, and Aligner, a human-aligned trained model that consumes the gathered evidence. This design fuses the breadth of evidence gathering with the discriminative judgment of human-aligned supervision.

In experiments on real-world submissions, RubricReviewer produced reviews that reviewers judged markedly more comprehensive and more discriminative than prior systems. The framework also exhibited the strongest robustness against adversarial prompt-injection attacks compared to baselines.

Ablation studies further confirmed the necessity of each component—Scout and Aligner—for achieving the reported improvements in review quality and robustness.