Skip to content
Agents · Nov 4, 2026

MEMENTO agent improves long-context QA accuracy from 36.9% to 67.8% while limiting context to 20K tokens

A Qwen3-8B-based agent trained with synthetic note-taking and RL outperforms a GPT-5.5 agent by 54.7 percentage points and nearly matches a ~100x larger GLM-5.3 agent at a fraction of the cost.

Trust79
HypeLow hype

1 source · cross-referenced

ShareXLinkedInEmail
TL;DR
  • MEMENTO, a long-context agent built on Qwen3-8B, raises accuracy on book-level QA from 36.9% to 67.8% while keeping context under 20K tokens.

Researchers from Columbia University and other institutions introduce MEMENTO, a long-context agent built on Qwen3-8B that learns context management through compaction. The approach combines fine-tuning on synthetic trajectories with notes generated in hindsight and reinforcement learning using a reward combining answer correctness, note-quality heuristics, and confidence calibration.

On book-level question answering across four benchmarks with books up to 2.5M tokens, MEMENTO improves Qwen3-8B’s accuracy from 36.9% to 67.8% while never holding more than 20K tokens in context. The agent also generalizes at test time to strategies such as context folding and confidence-based backtracking, even though it was trained only to compact into a fresh window.

In the same evaluation harness, MEMENTO outperforms a GPT-5.5 agent by 54.7 percentage points (67.8% vs. 13.1%) and exceeds a GPT-5.5 agent using full book context (56.7%) at less than 1/50 the cost. It nearly matches a ~100x larger GLM-5.3 agent (69.3%) at 1/9 the cost and comes within 6.4 points of GLM-5.3 full context performance at 1/26 the cost.

Sources
  1. 01arXiv cs.CL — MEMENTO: Learning Context Management for Long Context Processing Agents
Also on Agents

Stories may contain errors. Dispatch is assembled with AI assistance and curated by human editors; despite the trust-score filter, mistakes happen. We correct publicly — every article links to its revision history. Nothing here is financial, legal, or medical advice. Verify before relying on any claim.

© 2026 Dispatch. No ads. No sponsorships. No paid placement. Reader-supported via Ko-fi.

Built by a person who cares about honest AI news.