IBM Research study finds agentic memory dose must be calibrated to model capability
ALTK-Evolve experiments across eight models show strong models benefit from full guideline sets, weaker models need curated retrieval, and saturated models see no gain.
1 source · cross-referenced
- ALTK-Evolve distills reusable guidelines from an agent’s past trajectories and injects them at inference without model updates.
- Across eight models (30B to frontier proprietary), the optimal memory dose varies by capability tier: strong models want the full set, weaker models need curated retrieval, and saturated models show no measurable gain.
- On AppWorld’s 585 multi-step tasks, curated retrieval lifted gpt-oss-120b’s task completion +16.1pp at only +5% tokens; DeepSeek-V3.2 gained +9.5pp with the full set.
- The study defines three configurations—baseline (no memory), full guideline set, and curated retrieval—and reports both Task Goal Completion and stricter Scenario Goal Completion metrics.
IBM Research’s ALTK-Evolve framework distills reusable guidelines from an agent’s prior trajectories and injects them at inference time without updating model weights, enabling portable learning loops across different models.
The team evaluated ALTK-Evolve on AppWorld, a suite of 585 multi-step tasks across nine simulated applications, measuring both Task Goal Completion (TGC) and the stricter Scenario Goal Completion (SGC) metrics.
Across eight models spanning the capability spectrum—from a 30B dense model to frontier proprietary systems—the study observed three recurring patterns for agentic memory effectiveness.
Strong models with headroom, such as DeepSeek-V3.2 (671B MoE) and Claude Opus 4.6, benefited most from the full guideline set, achieving +9.5pp and +4.1pp gains in TGC respectively, with larger gains in SGC.
Weaker models like gpt-oss-120b (117B MoE) performed best with a curated retrieval strategy—combining a fixed high-confidence core with task-relevant guidelines—lifting TGC by +16.1pp while increasing tokens by only about 5%, thanks to prompt caching.
Already-saturated models, exemplified by GLM-5 (745B MoE), showed no measurable improvement from additional memory, indicating a ceiling effect on these tasks.
The study defines three configurations for comparison: a baseline with no memory, a full guideline set injected on every step, and a curated retrieval approach delivering a subset of guidelines per task.
The authors emphasize that the optimal memory dose depends on factors beyond parameter count, including benchmark headroom, context-window size, architecture, guideline quality, and task distribution, with ongoing work to disentangle these effects.
- Aug 17, 2026 · arXiv cs.CL
Researchers propose InflationAgent to cut agentic LLM costs by routing based on token inflation
Trust79 - Aug 15, 2026 · Latent Space — swyx
Flue 2 introduces React-style hooks for building dynamic agent harnesses
Trust78 - Aug 15, 2026 · AWS — Machine Learning Blog
AWS demonstrates multi-agent workflows combining SageMaker endpoints with Bedrock AgentCore
Trust79