Self-evolving LLM agent improves legal case retrieval without training via iterative rule refinement
Framework uses an LLM-based agent to autonomously generate, test, and refine query-rewriting rules, outperforming BM25 and human-designed baselines on LeCaRD-v2.
1 source · cross-referenced
- A new arXiv paper proposes a self-evolving agent that iteratively refines query-rewriting rules to improve legal case retrieval without parameter training.
- The framework pairs an LLM with an automatic evaluation environment to plan validation experiments and discard ineffective rules based on historical feedback.
- On the Chinese legal case retrieval benchmark LeCaRD-v2, the method outperforms non-evolutionary baselines, including human-designed rules and greedy rule selection.
- Results indicate the agent’s ability to leverage prior experimental outcomes and intrinsic rule-elimination knowledge is critical to refining the rule set.
Legal case retrieval is difficult because legal language is complex and requires precise lexical alignment between queries and relevant cases. While dense retrieval models have made progress, BM25 remains a strong baseline in this domain, motivating alternative approaches that do not require parameter training.
Researchers propose a self-evolving framework that equips an LLM-based agent with an automatic evaluation environment. The agent iteratively creates rewriting rules, plans validation experiments over rule combinations, and eliminates ineffective rules based on historical feedback.
The method is evaluated on the Chinese legal case retrieval benchmark LeCaRD-v2. Experimental results show the framework outperforms non-evolutionary baselines, including human-designed rules and greedy rule selection, particularly when powered by a high-capacity core LLM.
Analyses indicate the agent’s ability to leverage previous experimental results and its intrinsic knowledge of rule elimination are critical to refining the rule set via self-evolution.
- Jun 17, 2026 · arXiv cs.CL
Paper proposes PromptMN, a pseudo-prompting language to structure human-AI instructions
Trust79 - Jun 17, 2026 · arXiv cs.CL
Researchers propose MemSlides, a hierarchical memory framework for personalized slide generation agents
Trust79 - Jun 17, 2026 · arXiv cs.CL
Researchers propose RepSelect method to make LLM unlearning more robust against reversal attacks
Trust79