LLM theory of mind improvements on benchmarks don't reliably translate to better human-AI interactions, study finds

A new arXiv preprint questions whether improvements in large language models' theory of mind (ToM) capabilities—the ability to understand human beliefs, intentions, and perspectives—actually enhance real-world human-AI interactions. The researchers identified a methodological gap: existing ToM benchmarks rely on third-person, multiple-choice evaluation formats disconnected from the dynamic, first-person nature of actual conversations.

The team developed an interactive ToM evaluation paradigm and tested four prominent ToM enhancement techniques across multiple dimensions. Their evaluation covered both goal-oriented interactions (coding and mathematics tasks) and experience-oriented interactions (counseling scenarios), using both curated datasets and direct user studies with human participants.

The central finding contradicts a common assumption in AI development: performance gains on static benchmarks did not consistently predict improvements in interactive, open-ended dialogue settings. This discrepancy suggests that current evaluation methods may be optimizing for narrow test-taking ability rather than the nuanced social reasoning required in lived human-AI exchanges.

The authors argue that interaction-based assessments are essential for developing LLMs that can genuinely adapt to human needs and preferences. The work implies that labs pursuing ToM improvements should validate their techniques against realistic interaction scenarios, not just benchmark leaderboards.