Paper proposes stylistic triggers to bypass multimodal model safety defenses

A new arXiv preprint introduces Adversarial Style Optimization (ASO), a method to enhance jailbreak attacks on vision-language models (VLMs) by exploiting a stylistic inconsistency between comprehension and safety alignment. The authors argue that while VLMs can robustly understand content regardless of visual style, their safety mechanisms are vulnerable to specific stylistic triggers.

The proposed ASO is a plug-and-play module that fine-tunes an image-editing model to superimpose optimized stylistic modifications onto adversarial images. This process uses a Group Relative Policy Optimization (GRPO) agent guided by a Structurally-Tiered Reward Function, which combines a logit-based signal for detecting explicit refusals with a high-fidelity semantic evaluation from a powerful judge model.

The authors report that extensive experiments show ASO significantly enhances the attack success rate (ASR) of state-of-the-art jailbreak methods, indicating that stylistic biases represent a scalable vector for red-teaming VLMs.

The paper is accompanied by an open-source release of the code on GitHub, and the authors note that the work has been accepted for oral presentation at CVPR 2026.