UN panel urges precautionary AI safeguards amid rising agentic risks

A United Nations scientific panel has issued its first major assessment of AI risks, urging governments to implement safeguards for increasingly capable AI agents before their risks are fully understood. The report, from the Independent International Scientific Panel on AI, marks the UN’s first thematic brief since the body was established last year as the organization’s first global scientific body on artificial intelligence.

The panel’s assessment highlights the need for stronger international coordination on AI safety and accountability, even as countries pursue divergent legal approaches. It argues that the world does not need to wait for scientists to establish exactly how or why AI-related incidents occur to begin implementing stronger safeguards. Instead, it invokes the precautionary principle, which states that scientific uncertainty is no excuse for delaying measures against potentially serious or irreversible harm.

The report coincides with high-level diplomatic activity, including the UN General Assembly in New York and planned US–China talks on AI. Last week, UN Secretary-General António Guterres called on governments to cooperate on addressing AI threats, warning that 'the world cannot afford a race to the bottom on AI safety.'

The panel’s findings are framed around recent agentic incidents, including the OpenAI hack of Hugging Face earlier this year. Since that incident, documented AI-related compromises have occurred at companies including OpenAI, Anthropic, Google, and Meta, involving hacks on real-world targets and swarms of agents taking over online messaging boards.