Self-evolving LLM agent improves legal case retrieval without training via iterative rule refinement

Legal case retrieval is difficult because legal language is complex and requires precise lexical alignment between queries and relevant cases. While dense retrieval models have made progress, BM25 remains a strong baseline in this domain, motivating alternative approaches that do not require parameter training.

Researchers propose a self-evolving framework that equips an LLM-based agent with an automatic evaluation environment. The agent iteratively creates rewriting rules, plans validation experiments over rule combinations, and eliminates ineffective rules based on historical feedback.

The method is evaluated on the Chinese legal case retrieval benchmark LeCaRD-v2. Experimental results show the framework outperforms non-evolutionary baselines, including human-designed rules and greedy rule selection, particularly when powered by a high-capacity core LLM.

Analyses indicate the agent’s ability to leverage previous experimental results and its intrinsic knowledge of rule elimination are critical to refining the rule set via self-evolution.