Researchers propose InflationAgent to cut agentic LLM costs by routing based on token inflation
A new router measures how retry overhead inflates true workflow costs and selects models to maximize accuracy per predicted cost, reducing tokens used by 31% on GSM8K versus FrugalGPT.
1 source · cross-referenced
- Proposes InflationAgent, a four-stage router that explicitly models "token inflation" from retries in agentic LLM workflows.
- Measures inflation up to 4.25× for a 7B model on multi-hop QA and predicts it with AUROC 0.887 using a new signal, CoT Branching Entropy.
- On GSM8K under a fixed budget, achieves 94.7% accuracy vs 91.0% for FrugalGPT while using 31% fewer tokens.
- Validates a fresh-escalation policy: forwarding failed reasoning chains to GPT-4o can reduce its accuracy by up to 34.8 percentage points.
Researchers introduce InflationAgent, a four-stage router designed to account for "token inflation"—the gap between a model’s per-token price and the true cost of an agentic workflow that retries failed attempts. They define inflation as the ratio of true workflow cost to single-call cost and show that existing systems like FrugalGPT can underestimate real costs by more than twofold on difficult tasks.
The work quantifies inflation across model tiers and task types, reporting inflation as high as 4.25× for a 7B model on multi-hop question answering. To predict inflation before execution, the authors propose CoT Branching Entropy (CBE), a difficulty signal computed entirely from local inference that achieves an AUROC of 0.887 for predicting high inflation.
InflationAgent selects models by maximizing a Semantic Exchange Rate (SER) that divides expected accuracy by predicted true cost, and it implements a fresh-escalation policy that discards failed reasoning chains before routing to a stronger model. On GSM8K under a fixed budget, the router attains 94.7% accuracy compared with 91.0% for FrugalGPT while using 31% fewer tokens.
The authors also evaluate the fresh-escalation policy by measuring the impact of forwarding failed reasoning chains to GPT-4o, which can reduce its accuracy by up to 34.8 percentage points, supporting the design choice to reset chains rather than carry forward partial results.
- Nov 11, 2026 · Latent Space — swyx
OpenAI demonstrates voice steering, cloud handoffs, and personal agents for Codex at AI Engineer Code Summit
Trust79 - Nov 11, 2026 · The Verge — AI
OpenAI to launch verified AI Business Agents in ChatGPT for US e-commerce and service businesses
Trust79 - Nov 10, 2026 · Latent Space — swyx
NaiveAI releases Naive-N0.5-Flash coding model with ultrafast token throughput
Trust74