Researchers propose Elastic Latent Recurrence to unify recurrent and non-recurrent language models without inference-time trade-offs

A latent recurrent language model typically incurs a ‘one-loop tax’—a performance gap at one loop compared to a non-recurrent model of the same size—making it less attractive for latency-sensitive deployments. Researchers propose Elastic Latent Recurrence (ELR), a training recipe that eliminates this tax by combining three techniques: a gated state update with a learned step size, loop counts mixed within each optimizer update, and conditioning each pass on its loop index and budget.

The first two techniques preserve one-loop accuracy, while the third reduces loss at every additional loop. Mixed depths can be introduced late in pretraining to limit added training cost, and with zero-initialized conditioning added at the switch, this late schedule achieves lower loss than training from scratch with mixed depths.

In Mixture-of-Experts (MoE) models, reusing first-loop expert routes on later loops reduces mixed-depth update time by 11.7% without measurable accuracy loss. At inference, retaining only the last loop’s attention KV cache maintains attention KV memory at one-loop levels while preserving most gains from additional loops.

On a 4B dense model and a 30B-A3B MoE hybrid, ELR at one loop is 0.52 points below and 1.04 points above matched baselines on a ten-task average, respectively. Four loops add 7.25 and 4.75 points. Applied to post-training, the recipe yields Koi-4B, a single checkpoint scoring an estimated 18 on the Artificial Analysis Intelligence Index at one loop (matching Qwen3.5-4B) and 29 at four loops.