Researchers identify why language models overuse external tools instead of relying on internal knowledge

A research team led by Harbin Institute of Technology has documented a widespread yet underexplored behavior in large language models: they routinely invoke external tools and APIs even when equipped to answer questions using their own training data. The team tested this phenomenon across multiple model sizes and architectures, confirming it is not isolated to a single system.

The researchers identified two distinct mechanisms driving this behavior. First, models appear to develop what they call a 'knowledge epistemic illusion'—they systematically misidentify the boundaries of their own knowledge. When presented with questions they could answer internally, models failed to recognize this capability and defaulted to external tool calls. The team proposed a corrective training approach using direct preference optimization that realigns models' self-assessment of their knowledge. In tests, this method reduced unnecessary tool usage by 82.8 percent while maintaining or improving accuracy.

The second mechanism traces to training incentives. Models trained with outcome-only rewards—where only final answer correctness matters—learned to treat tool usage as a safety net regardless of whether it was necessary. When researchers introduced balanced reward signals that explicitly penalized redundant tool calls, models with 7 billion and 32 billion parameters cut unnecessary tool invocations by 66.7 percent and 60.7 percent respectively, without sacrificing answer quality.

The paper includes theoretical analysis supporting both findings and suggests these insights could inform more efficient training regimes for tool-augmented systems.