Frontier labs and hedge funds outline controls for safe financial agents

OpenAI’s Joseph Kim described a "threshold of usefulness" where models become reliable enough to delegate tasks like investment idea generation and trading, predicting these capabilities will be viable in 2027. He emphasized traceability in financial workflows, arguing that review should be a natural part of the agent experience rather than an afterthought.

Anthropic outlined a "trust layer" for financial agents, including identity and permissions, sandboxing, and approval layers. Keon Shahab of Anthropic described a trade break investigation agent that proposes amendments for human approval, illustrating the boundary between proposing and acting. The framework aims to enable deployment at scale while maintaining control.

Two Sigma’s John Feminella argued that safety and innovation are not opposing goals, criticizing "approval fatigue" where excessive human oversight can both stifle innovation and introduce risk. He described composable controls—such as auto-mode classifiers that route only high-impact requests to humans—as a way to balance autonomy and safety.

Bloomberg’s Andrey Rybka shared real-world incidents where agents caused operational issues, including one where an agent misinterpreted instructions in a marketplace review as a directive to stock a product. He recommended external controls like scoped identities, tool call restrictions, audit trails, budgets, kill switches, and input validation to mitigate risks outside the model.