Apple study finds Plan-then-Retrieve outperforms Retrieve-then-Plan for multi-tool agent queries

Apple’s Machine Learning Research group introduced Plan-then-Retrieve (PtR), an alternative to the prevailing Retrieve-then-Plan (RtP) paradigm for tool-augmented LLM agents. In RtP, the model receives the user query and retrieves relevant tools via approximate nearest-neighbor search, whereas PtR first asks the model to generate Abstract Tool Stubs—hypothesized function signatures that describe the needed capabilities—and then uses those stubs as retrieval queries against the tool registry.

The team evaluated both paradigms on ToolBench, which contains 46,984 unique API endpoints across 50 categories, using three progressively harder query subsets: single-tool (G1), intra-category multi-tool (G2), and cross-category multi-tool (G3). They observed a complexity-dependent crossover: RtP outperforms PtR on single-tool queries, while PtR outperforms RtP on multi-tool queries. This pattern held across every retrieval budget tested (k = 1 to 20) and across registry sizes ranging from 1,000 to 47,000 APIs, and was consistent across two model families (Gemini 2.5 Flash and gpt-oss-120b).

The authors characterize a stub-hallucination failure mode in PtR, where the model invents plausible but non-existent API names. On simple queries, these hallucinations can remove proper-noun signals that RtP matches directly, hurting performance; on complex queries, the same mechanism helps decompose multi-faceted intent into functional sub-needs, improving performance.

They further analyze token overhead and show that PtR’s extra discovery call roughly doubles inference cost compared to RtP. This additional cost pays off on multi-tool queries but harms efficiency on single-tool ones, leading the authors to propose complexity-adaptive routing as a practical direction. They suggest using RtP’s top-1 retrieval score as a zero-cost signal to drive routing decisions.