CoFiBreak method improves diversity of jailbreak attacks while maintaining high success rates in red-teaming LLMs
Researchers propose a coarse-to-fine diversity reward mechanism for reinforcement learning-based red teaming that increases the variety of successful attack strategies without sacrificing effectiveness or inference efficiency.
1 source · cross-referenced
- CoFiBreak introduces a two-level diversity reward to broaden attack strategies and their realizations in automated red teaming of LLMs.
- On HarmBench, CoFiBreak discovers 1.6–2.0x more distinct successful fine-grained strategies than the strongest baseline across frontier victim models.
- On AdvBench, CoFiBreak achieves 98.8% ASR@10 with the highest diversity score, 2.1x that of the strongest-attacking baseline.
- Attacks transfer to unseen frontier LLMs, reaching 71.0% ASR@10 on AdvBench with the unseen victim Claude-Opus-4.6.
Automated red teaming uses reinforcement learning to train an attacker LLM to elicit harmful responses from a victim LLM, but optimizing only for attack success can cause the attacker to collapse onto a narrow set of prompts, reducing the diversity of tested behaviors.
CoFiBreak addresses this by adding a coarse-to-fine diversity reward to Group Relative Policy Optimization (GRPO), which broadens the strategies the attacker employs and how those strategies are realized in prompts.
The coarse reward encourages underrepresented strategies within each GRPO group, while the fine reward discourages reuse of the same personas, framings, and output formats after harmful-intent information is removed from prompt embeddings.
Diversity rewards are applied only to attacks that already exceed an effectiveness threshold, ensuring that gains in diversity do not come at the cost of attack success.
Because both rewards are computed during training, CoFiBreak retains inference efficiency, avoiding additional runtime overhead.
In experiments with Llama3.1-8B-Instruct as the victim, CoFiBreak discovered 1.6–2.0x as many distinct successful fine-grained strategies as the strongest baseline across frontier victims on HarmBench.
On AdvBench, CoFiBreak reached 98.8% ASR@10 (attack success rate with 10 generations per intent) with the highest diversity score among all compared methods, which was 2.1x that of the strongest-attacking baseline.
The method also transferred to unseen frontier LLMs, achieving 71.0% ASR@10 on AdvBench with the unseen victim Claude-Opus-4.6.
- Nov 11, 2026 · Schneier on Security
Three arrested in multi-country scheme using AI to impersonate executives and lawyers in $129M fraud
Trust79 - Nov 10, 2026 · Ars Technica — Technology Lab
Grok Build link-sharing design exposes 35,000 apps and sensitive user data
Trust79 - Nov 10, 2026 · Schneier on Security
OpenSSH vulnerability allows login to password-disabled Alpine Linux accounts via oversized password
Trust79