Apple proposes PROOF-Gen to recover failed teacher trajectories for better model distillation
Method converts near-miss failures into corrective guidance, boosting trajectory quality and downstream task performance across benchmarks and locales.
1 source · cross-referenced
- Apple’s ML Research introduces PROOF-Gen, a method to recover correct trajectories from failed teacher demonstrations by generating per-scenario corrective guidance.
- On τ 2-bench, PROOF-Gen recovers 93% of previously failed scenarios.
- Fine-tuning with the recovered data improves Qwen3-4B-Instruct-2507 from Pass^1=0.132 to 0.529 and Gemma 4 E4B-it by +7.2pp on BFCL v4 multi-turn.
- In a deployed pipeline, trajectory quality improves by +6.3pp goal completion and transfers to an on-device model with gains up to +5.0pp across response-quality metrics.
Apple’s Machine Learning Research team describes PROOF-Gen (Per-scenario Reflective Optimization to Overcome Failed Generation), a technique that transforms failed teacher-generated trajectories into corrective guidance for the teacher model. The method analyzes execution traces and evaluation feedback for each failed task, then writes task-specific corrective instructions that steer the teacher toward a passing trajectory. These instructions are removed before training, ensuring the student learns from clean demonstrations without task-specific scaffolding.
The authors report that on τ 2-bench, 57% of teacher trials fail, with two-thirds of those being near-misses—cases where most tool calls are correct but undone by a single decisive error. PROOF-Gen recovers 93% of these failed scenarios by generating per-scenario corrective guidance.
When fine-tuning models on the combined dataset of original and recovered trajectories, Qwen3-4B-Instruct-2507 improves from Pass^1=0.132 to 0.529, and Gemma 4 E4B-it gains +7.2 percentage points on BFCL v4 multi-turn. In a deployed pipeline, the method increases trajectory quality by +6.3 percentage points in goal completion and transfers to an on-device model with gains ranging from +1.7 to +5.0 percentage points across response-quality metrics. Positive transfer is observed in every locale, with non-English averages improving by +1.48 percentage points.
- Oct 13, 2026 · Google DeepMind — Blog
Google DeepMind details safety guardrails for Gemini 4 Argon’s cyberdefense and biology capabilities
Trust76 - Oct 13, 2026 · Google DeepMind — Blog
Google DeepMind unveils Gemini 4 Argon for robotics with 87% score on Embodied Reasoning benchmark
Trust78 - Oct 13, 2026 · arXiv cs.CL
Researchers propose Elastic Latent Recurrence to unify recurrent and non-recurrent language models without inference-time trade-offs
Trust79