HALO method improves frozen language models with adaptive refinement, outperforming fixed baselines on MMLU-Pro and GPQA-Diamond
Hybrid adaptive latent-refinement approach achieves best paper-facing average on combined MMLU-Pro and GPQA-Diamond benchmarks while reducing controller compute relative to fixed-refinement baselines.
1 source · cross-referenced
- HALO introduces a hybrid adaptive latent-refinement method to improve frozen pretrained language models with minimal extra compute.
Researchers propose HALO, a hybrid adaptive latent-refinement method designed to improve frozen pretrained language models with a small amount of adaptive extra computation. Unlike fixed refinement strategies that either underperform or waste compute, HALO combines a coarse refinement stage with selective second-stage latent refinement applied only to a subset of tokens chosen via token scoring and monotonic token halting.
On a combined benchmark derived from MMLU-Pro and GPQA-Diamond, HALO achieves the best overall average among paper-facing methods, surpassing the frozen backbone and both fixed-1 and fixed-2 baselines. Internal analysis indicates HALO reaches nearly the same token-accuracy level as fixed-2 while using fewer average applied refine steps than fixed-1 and far fewer than fixed-2.
The authors argue that the advantage of HALO stems not from simply adding more refinement steps, but from better allocating refinement effort across tokens. This yields the strongest paper-facing result while reducing measured controller compute compared to either fixed baseline.
- Nov 11, 2026 · The Verge — AI
Google DeepMind unveils AlphaGenome 2 to predict genetic variant effects on cell types
Trust79 - Nov 11, 2026 · Google DeepMind — Blog
Google DeepMind unveils AlphaGenome 2 to predict DNA variant effects in specific cell types
Trust84 - Nov 11, 2026 · arXiv cs.AI
Researchers demonstrate auditable human-agent teams for autonomous synchrotron X-ray experiments
Trust79