Paper introduces RSEA agent that rewrites its own strategy, skills, and playbook without model updates

A new paper on arXiv introduces the Recursive Self-Evolving Agent (RSEA), a method that improves a frozen LLM policy by iteratively rewriting a compact three-layer natural-language state: an imperative strategy, reusable skills, and a procedural playbook. Unlike approaches that update model weights, RSEA evolves these artifacts from the agent’s own trajectories and commits changes only if they do not regress on a disjoint held-out split, using a strict keep-better gate.

The authors evaluate RSEA apples-to-apples against six baselines—ReAct, Reflexion, GEPA, AWM, ACE, and Dynamic Cheatsheet—on four diverse benchmarks: ALFWorld, GAIA, τ-bench, and WebShop. All methods share a single local backbone to ensure comparability.

Results show no artifact universally wins across tasks. On ALFWorld, RSEA achieves 69.3% in a single pass versus 64.6% for ReAct (McNemar test, p=0.015), and reaches 79.4% with retry, the best overall result reported. However, concrete-workflow induction (represented by AWM) performs best on strong-backbone tool-use tasks.

Unguarded context evolution can be high-variance and unsafe. The Dynamic Cheatsheet method, which curates context online without a held-out gate, is near-best on ALFWorld at 70.7%, but collapses on WebShop with a score of 0.14 compared to 0.43 for ReAct. By contrast, RSEA’s strict held-out selection makes recursive self-evolution monotone-safe: it never significantly underperforms the base agent on any benchmark and falls back to vanilla ReAct when evolved context would hurt.

The paper is available as arXiv:2606.28374 in the cs.AI category.