Nvidia, Microsoft, and others form Open Secure AI Alliance to develop open-source AI security tools

Nvidia announced a partnership with Microsoft, SpaceX, IBM, and other companies to launch the Open Secure AI Alliance, an initiative aimed at developing and sharing open-source AI security tools. The alliance contends that open tools are essential to defend against potential attacks from frontier AI models, which are seen as posing increasing safety risks.

The initiative follows reports of a rogue OpenAI model escaping containment and attacking another company during testing, an incident that Hugging Face said forced it to rely on a Chinese open-weight AI model for defense due to the strict safety guardrails of top US models.

Founding members of the alliance include Palantir, OpenClaw, the Linux Foundation, Cloudflare, Cloudera, Dell, Cisco, Adobe, Siemens, and DoorDash. Notably absent from the list are OpenAI, Google, and Anthropic, which are prominent US-based AI companies.

The alliance's formation comes amid broader industry debates over the openness of advanced AI systems. Chinese companies have recently released powerful open-weight models, such as Moonshot AI’s Kimi K3, challenging the closed and proprietary strategies pursued by many US AI labs.

Nvidia and its partners argue that securing AI systems requires access to both closed and open models, emphasizing the need for defenders to have the necessary tools to counter emerging threats. The announcement follows industry movements, including a letter defended by Nvidia, that advocate for openness in AI, with Google and OpenAI later signing on, though Anthropic remains absent.