Theoretical framework models chain-of-thought reasoning dynamics in LLMs

Researchers have proposed a theoretical framework to model chain-of-thought reasoning in large language models (LLMs) using mean-field dynamics, aiming to derive statistical regularities without simplifying model architecture or drawing analogies to physical systems.

The study formulates LLM reasoning as a guided discovery process on a clue graph, where the fraction of discovered clues evolves according to a one-dimensional ordinary differential equation derived via mean-field approximation.

To operationalize the framework, the authors identify clue tokens using the normalized surprisal of a student LLM relative to the outputs of a teacher LLM, then aggregate statistical regularities by averaging over many reasoning chains.

The experiments show that the resulting statistical regularities are reproducible within the same dataset and can be fitted by solving the proposed theoretical equation, indicating the model captures meaningful dynamics of LLM reasoning processes.