Microsoft Research Asia open-sources Agent Lightning v1.0, a lightweight framework for training AI agents with reinforcement learning

Microsoft Research Asia has open-sourced Agent Lightning v1.0, a lightweight framework designed to simplify reinforcement learning (RL) for AI agents by connecting existing agent harnesses directly to RL training without requiring reimplementation.

The framework introduces Harnessed Agentic RL, a training paradigm in which the same agent harness used in deployment participates directly in RL, removing the need to rebuild the agent inside the training framework. This approach contrasts with traditional agentic RL, where the training framework manages the environment and agent loop, often requiring costly reimplementation of the agent.

Agent Lightning v1.0 is implemented in roughly 3,500 lines of code and supports native Kubernetes deployment, allowing agents to run as standard Kubernetes jobs on self-managed clusters, cloud Kubernetes, or local infrastructure without dependency on paid commercial sandbox services.

The framework includes a complete, reproducible agent RL training pipeline and provides an end-to-end coding agent example using Qwen3.5-9B, which achieved a Pass@1 score of 56.4% on SWE-bench Verified—an absolute improvement of 14.6 percentage points—using approximately 6,000 training samples based on an open-sourced dataset.

In Harnessed Agentic RL, an LLM proxy sits between the agent and the model, enabling the training framework to observe and record model calls without altering the agent's existing harness or execution logic. This design allows the deployed agent harness to participate directly in RL training, preserving its behavior and reducing development overhead.

The researchers highlight four challenges introduced by training with real harnesses: retokenization and sample merging, advantage calculation, loss normalization, and training backend scheduling. These challenges stem from the variability in how harnesses manage context and produce training samples, which can differ from the assumptions of traditional RL frameworks.