Researchers propose Generalized Agent Iteration to unify iterative policy improvement and recursive self-improvement under one formal framework

A new paper on arXiv proposes Generalized Agent Iteration (GAI), a formal framework intended to unify iterative policy improvement and recursive self-improvement (RSI) under a single learning paradigm. The authors argue that while iterative policy improvement has a well-established classical framework—generalized policy iteration (GPI)—no analogous formalism exists for RSI despite numerous claims of self-improvement across scales.

In GAI, an agent is defined as a configuration of modifiable components within a system, and learning is modeled as a cycle of agent evaluation followed by agent improvement. The framework introduces two pivotal distinctions: whether the mechanism responsible for improvement is part of the agent itself, and whether the standard used to evaluate the agent is grounded outside the agent. These distinctions delineate classical iterative policy improvement from RSI and determine a system’s polarity as anchored, goal drift, or fully self-referential.

The authors use these two axes to place existing systems on a common coordinate system and argue that this enables defects of recursive self-improvement to be stated and analyzed one condition at a time. They position GAI as a first step toward a formal characterization of RSI that is grounded in classical accounts, makes existing systems comparable, and provides a principled basis for analyzing and designing new autonomous and evolving intelligence systems.