Apple researchers propose method to distinguish sources of annotation disagreement in AI safety policies

Apple’s Machine Learning Research team proposes Annotator Policy Models (APMs), a method to learn interpretable representations of individual annotators’ internal safety policies directly from their labeling behavior. The approach avoids the cost and reliability issues of asking annotators to self-report their reasoning, which often fails to reflect actual decision processes.

The researchers validate APMs by demonstrating they can accurately model annotator safety policy with over 80% accuracy, faithfully predict how annotators would respond to counterfactual edits to safety policies, and recover known policy differences in controlled experimental settings.

Using APMs, the team applies the method to both human and LLM annotators to surface two core applications: identifying policy ambiguity by revealing how annotators interpret safety instructions differently, and surfacing value pluralism by uncovering systematic differences in safety priorities across demographic groups.

The work was accepted for presentation at the Principled Design for Trustworthy AI — Interpretability, Robustness, and Safety across Modalities Workshop at ICLR 2026, and builds on prior research such as SafetyPairs and Policy Maps, which address related challenges in AI safety and policy design.