Researchers propose GAD-RL to improve OCR faithfulness in vision-language models

Vision-language models frequently rewrite anomalous or noisy text in images into linguistically plausible but incorrect transcriptions, compromising OCR faithfulness. Sequence-level task rewards and local teacher guidance are complementary approaches, but fixed teacher supervision becomes less effective as the student model improves during training.

To address this, the authors introduce Gated and Attenuated On-Policy Distillation (GAD-RL), a method that adaptively regulates teacher supervision during joint post-training based on the student’s current task performance and local response distributions. A frozen teacher model conditions on reference transcriptions and student-generated prefixes to provide guidance.

GAD-RL implements two key mechanisms: it disables distillation entirely for response groups containing an output with a task reward of at least 0.95, and it continuously attenuates distillation strength as the mean reward of a response group increases. Additionally, it weights the forward KL divergence by the student’s probability of the teacher’s top-1 token, reducing the influence of local auxiliary updates when the student’s confidence in the teacher’s candidate is low.

The approach is evaluated on Qwen3.5-2B, where GAD-RL achieves 59.92% Micro Recall on CHAOS-Bench, surpassing GRPO by 8.45 percentage points and GRPO+OPD (fixed-weight) by 4.43 percentage points. On OmniDocBench v1.6, GAD-RL achieves an Overall score of 91.18.