Theory shows in-context search with reflection can exponentially reduce sampling complexity for reasoning tasks

A new arXiv preprint proposes a theoretical framework to analyze when in-context search—iterative generation, critique, and revision of solution attempts—improves reasoning performance in large language models (LLMs). The authors model in-context search as approximate inference over reasoning traces, where the base model defines a prior distribution over solutions and self-reflection provides feedback used to update a posterior distribution over correct answers.

The paper shows that when reflections reliably identify and localize early mistakes, in-context search can yield exponential improvements in sampling complexity compared to zero-shot inference. Specifically, problems with exponentially small zero-shot pass rates can be solved using only a polynomial number of sequential attempts, whereas without reliable reflection, conditioning on past attempts offers no asymptotic benefit over parallel sampling.

The authors further prove that these gains are robust and learnable. They demonstrate that approximate posterior updates are sufficient for the exponential improvement, and that cross-entropy training on search rollouts can recover the required reflection behavior with polynomial sample complexity. This suggests that the benefits of in-context search can be induced through training rather than requiring perfect inference-time reflection.

Finally, the paper connects the framework to reinforcement learning by showing that, under a stagewise abstraction with verifiable rewards, the optimal policy extension implements the same posterior reweighting rule as the in-context search mechanism. The authors validate key qualitative predictions of the theory using experiments on large reasoning models, supporting the practical relevance of the theoretical results.