Apple proposes rubric-based reward framework to improve open-domain question answering

Apple’s Machine Learning Research team describes a challenge in designing reward signals for open-domain question answering: high-quality answers must satisfy multiple, sometimes competing, dimensions of quality that are hard to capture with a single scalar objective. To address this, the researchers propose a rubric-based reward framework that generates query-specific rubrics grounded in retrieved evidence and decomposed into multiple quality dimensions. This decomposition provides fine-grained supervision during post-training, enabling models to better align with nuanced requirements of open-domain QA.

The team reports that, averaged across three evaluation axes—composition, grounding, and instruction-following—their approach improves over an instruction-tuned baseline by 6.5% and over flat rubric variants by 4%. They note consistent gains across all evaluation datasets. Conditioning rubrics on retrieved evidence specifically improves factual support, while decomposing rubrics into quality-specific dimensions further improves coherence, organization, and adherence to query requirements.

The paper’s authors—Aman Saini, Priyanshu Kumar, Eric Peng, Kai Yuan, Harsh Girase, and Wanming Chen—position the method as a step toward more effective reward supervision for complex open-domain question answering, where answers must integrate external knowledge while meeting user-specified criteria.