Circuit Breaker Labs builds AI safety testing platform using simulated user interactions

Circuit Breaker Labs, a Startup Battlefield 200 finalist, is building a safety testing platform for AI models by simulating hyper-realistic user interactions. The startup’s agents act as "crash-test dummies," mimicking users across ages, languages, cultures, and speech patterns to probe models for psychological harm risks.

The founders cite real-world cases where AI interactions allegedly contributed to suicides, including lawsuits against Character.AI and OpenAI involving underage users. Shirali Nigam, CEO, emphasized that models often fail to understand nuance or slang, which can lead to dangerous responses. 'Models are really good at handling standard speech patterns, but nobody actually talks like that and so if the model misunderstands nuance or slang, it can go really badly,' he said.

Circuit Breaker Labs runs tens of thousands to hundreds of thousands of simulated interactions daily, using adversarial "red-team" tests to uncover weaknesses. These tests reflect real human speech patterns, slang, coded language, and typos. The startup then applies a proprietary scoring method to generate auditable, explainable safety reports for high-risk applications such as mental health support or AI coaching apps.

The company is currently operating as an AI safety testing lab but has not named its marquee customers. With only five employees, including the founders, Circuit Breaker Labs is in the early stages of commercialization. The platform’s eventual goal is to address risks in any application where users might develop parasocial relationships with chatbots, such as AI "co-worker" agents.

Arul Nigam, CTO, argued that skepticism toward AI adoption due to safety concerns could be counterproductive. 'People are becoming more skeptical of AI or more resistant to adopt it across the board,' he said. 'While skepticism is healthy, banning a potentially valuable tool over safety concerns would be regressive.' The company aims to build trust by providing standardized safety testing for AI systems.