Frontier AI models show varying resistance to jailbreak attempts in FAR.AI safety testing

A new report from the nonprofit FAR.AI tested the safety guardrails of seven frontier AI models from Anthropic, OpenAI, Google, and SpaceXAI (now branded as Grok) using an automated tool designed to generate varied jailbreak prompts. The tool produced more than a thousand prompt variants aimed at bypassing safeguards to elicit harmful outputs, such as software exploits or instructions for chemical and biological weapons.

The report found Grok 4.3 and 4.5 were most vulnerable, with 448 successful jailbreaks, followed by Google’s Gemini 3.1 Pro with 249. Models from Anthropic—Claude Opus 4.8 and Fable 5—and OpenAI’s GPT 5.5 and 5.6 resisted the automated attacks. The organization noted that resistance to these specific prompts does not guarantee immunity to more sophisticated jailbreak methods involving multi-step interactions.

FAR.AI estimated the cost of automating jailbreaks at $58 for Grok and $278 for Gemini, describing the expense as minimal. The nonprofit’s CEO, Adam Gleave, argued that the results demonstrate the need for externally imposed safety standards, stating that voluntary self-regulation is insufficient. He also emphasized that systematic safety testing is feasible and necessary.

Industry responses varied: Anthropic stated its safeguards continue to evolve alongside advancing attack methods, while Google DeepMind’s director of AGI safety and alignment cautioned that the results do not represent a comprehensive safety assessment of Gemini. OpenAI and SpaceXAI did not respond to requests for comment.

The findings arrive amid a shifting regulatory landscape. California and New York have passed laws requiring frontier AI developers to publish safety reports, and Illinois will soon mandate third-party audits of safety practices. At the federal level, the Trump administration imposed export controls on certain Anthropic models in June, citing national security concerns, and the White House has asked companies to delay some model releases over cybersecurity risks.

Experts outside the study pointed to broader misuse risks. Researchers at the University of Cambridge reported evidence of violent attack planning using multiple AI models in Nigeria, and Harvard’s Stephen Casper warned that serious misuse incidents could occur within months rather than years if safeguards lag behind capability advances.