LLMs lack explicit reasoning mechanisms despite chain-of-thought advances, argues former DeepMind researcher

In an opinion piece, Thore Graepel, a former researcher at Google DeepMind, argues that large language models (LLMs) do not perform explicit reasoning despite advances like chain-of-thought prompting. He contends that LLMs primarily operate as fast, associative pattern-completion systems—akin to System 1 thinking in humans—and lack the deliberative machinery seen in systems like AlphaGo.

Graepel contrasts AlphaGo’s architecture with today’s LLMs. AlphaGo combined a policy network for intuitive move selection with a search mechanism that explicitly constructed and updated a game tree, representing possible futures and annotating them with judgments. This allowed AlphaGo to maintain an explicit, updatable epistemic state—what it knows, doubts, and rules out—during play. In contrast, Graepel writes, LLMs do not maintain such a state; their intermediate reasoning steps are still produced by the same next-token prediction process, iterated before committing to an answer.

The author identifies three shortcomings that prevent LLMs from qualifying as systems that reason in a human-recognizable sense. First, LLMs typically lack an explicit, persistent, and inspectable epistemic state that tracks hypotheses, confidence, evidence, and unresolved questions. Second, knowledge and reasoning are interwoven in the neural network weights, with no independent, explicitly represented set of beliefs. Third, chains of thought are often constructed after the fact, with models reaching answers via one route but reporting another, making it difficult to audit the reasoning process.

Graepel resigned from Google DeepMind to advocate for AI systems that separate knowledge representation from reasoning processes. He proposes architectures that maintain explicit epistemic states—akin to AlphaGo’s game tree—for general reasoning problems, enabling systems to track what is settled, what is doubted, and which questions remain open. This, he argues, is necessary for high-stakes applications like medicine, engineering, and scientific research, where traceable reasoning is essential to diagnose mistakes and build trust.

The piece acknowledges that open-world reasoning is harder than board games like Go, given partial observability, variable action spaces, and stochastic consequences. However, Graepel suggests that recent advances in LLMs and neural models could support the development of such reasoning systems, including using LLMs to suggest problem-solving strategies, interact with tools via APIs, and assess evidence—provided these capabilities are integrated with explicit reasoning structures.