Researchers propose AutoIndexer for post-generation editing of LLM outputs via chains of edits

Researchers from Autodesk AI Lab introduce AutoIndexer, a generative model designed to address a limitation of autoregressive decoding: the inability to revise outputs after generation for non-causally ordered data such as cross-references or future dependencies. Unlike standard autoregressive or diffusion models, AutoIndexer learns to modify its prior outputs in hindsight through a sequence of token insertions, substitutions, and deletions, simulating a text editor’s cursor and edit operations.

The model’s training relies on a Chain of Edits mechanism, where ground-truth labels are perturbed using inverse chained edit operations. The model is then trained to undo these perturbations, learning to predict marker tokens that open and close edits while moving a cursor to specific positions in the existing token sequence. This training strategy is intended to make the model robust to erroneous context and adaptable to editing tasks such as code correction.

The authors report that AutoIndexer exhibits post-generation context-editing capabilities that most autoregressive and diffusion models lack, without compromising performance on standard LLM benchmarks. They also release the code and associated demos, including implementations on Hugging Face Spaces and Replicate.

The work is presented as an arXiv preprint (arXiv:2610.12545) and includes contributions from Shu Ishida, Aliasghar Khani, Tianyu Zhang, The Cong Luong, James Seale Smith, and Adam Gaier.