Researchers propose Core–Tail World Model to address heavy-tailed memory use in long-horizon language agents

Long-horizon language agents increasingly rely on external memory as a frozen world model, but current systems are typically judged only by task success or token cost. Researchers argue that the missing object is the shape of memory use: under finite context and repeated retrieval, agent memory can concentrate on a small core while leaving rare states in a long tail where prediction errors accumulate.

The team conducted a conservative tail audit and found that concentration is reproducible but policy-dependent. Random-walk agents produce log-normal-compatible retrieval artifacts, whereas semantic LLM policies yield the strongest truncated-power-law-compatible core–tail traces.

Motivated by these findings, they propose the Core–Tail World Model (CTWM), a rank-based memory controller that allocates prompt budget with a single exponent τ while retaining a summarized tail.

On Synthetic Graph World, CTWM preserves full state and transition coverage, reduces prompt tokens by 5.9%, and lowers bottom-half tail prediction error by 13.6% relative to a graph-memory baseline. The same paired comparison yields consistent token savings on ALFWorld and a 24.48% token reduction on LongMemEval with aggregate accuracy parity.

The results suggest that heavy-tailed memory traces are not only a diagnostic of finite retrieval, but also a practical control signal for token-efficient agent world models.