Microsoft Research proposes EvoLib, a framework that lets models learn from experience during inference without fine-tuning

Microsoft Research describes EvoLib as a framework that enables large language models to learn from their own experience during inference, without requiring ground-truth labels or external feedback. The approach treats raw experiences as sources for extracting reusable skills and reflective insights, which are then refined into increasingly general knowledge over time.

Unlike traditional memory systems that store static experiences, EvoLib consolidates new knowledge with similar existing knowledge, creating more general and reusable skills. It also dynamically reweights the importance of each knowledge unit based not only on immediate utility but also on contributions to future tasks, elevating knowledge with the greatest long-term impact.

The team evaluated EvoLib across mathematical reasoning, constrained code generation, and long-horizon decision-making tasks. In each setting, EvoLib outperformed top retrieval-based memory approaches and other abstract memory mechanisms while using tokens more efficiently. The framework also converted test-time compute into performance gains more rapidly than compute-scaling methods that treat each task in isolation.

EvoLib is designed to work with any black-box language model or AI system deployed via APIs, meaning it can be applied without model updates. The authors suggest this capability could be particularly valuable for deployed agents that need to adapt to new tasks over time without retraining.