AWS SageMaker AI adds multi-turn reinforcement learning for fine-tuning search agents

Amazon SageMaker AI now supports multi-turn reinforcement learning (MTRL) to fine-tune LLM-powered search agents, enabling optimization across full multi-turn trajectories rather than isolated steps.

The new capability addresses limitations of traditional approaches: supervised fine-tuning requires costly expert demonstrations, while single-turn reinforcement learning (e.g., RLVR) scores responses individually, ignoring interdependent decisions across turns.

MTRL frames agentic tasks as sequences of decisions, uses multi-turn rollouts to generate training data, and applies policy gradient algorithms for optimization, according to AWS’s description.

SageMaker AI MTRL offers a modular agent-environment interface for low-code integration, custom rewards, and custom tool loops, as well as serverless execution for production-scale training at per-token pricing without GPU cluster management.

AWS claims the approach yields gains in retrieval quality and reliability for search agents, attributing this to the model’s ability to learn environment-specific behavior and optimize across full sequences of interactions.

The post emphasizes that fine-tuning a smaller model with MTRL can deliver the reliability typically associated with frontier models while maintaining the speed and cost efficiency of smaller models.