Vals raises $40M Series A led by Andreessen Horowitz to build neutral AI benchmarking

Vals, a two-year-old startup, says legacy AI benchmarks are outdated and can be gamed by companies that train their models on publicly available tests. To address this, Vals does not publicly disclose its evaluation materials, aiming to reduce the risk that models are optimized for specific benchmarks rather than general capability.

The company evaluates models on tasks tied to specific industries—such as law, finance, and coding—as well as newer domains like recursive self-improvement, mental health, cybersecurity, biosecurity, and the law of armed conflict. According to Rayan Krishnan, Vals’ 25-year-old co-founder, the goal is to measure not just abstract knowledge but whether models can produce outputs comparable in quality to humans within each domain.

Vals also assesses potential negative outcomes, including how models might behave if deployed without safeguards. Krishnan describes this as evaluating “if these models ran wild in the world, what the negative implications would be.”

Clients pay Vals to run these evaluations, a model the company compares to paying the College Board to take the SAT. Krishnan argues that rigorous, neutral evaluations help companies troubleshoot and improve their models over time. These assessments are increasingly used by buyers to inform purchasing decisions, the company says.

Vals’ revenue has grown eightfold year-over-year, and its staff has tripled from eight to 25 since January. The company plans to expand headcount by an additional 10 to 15 people and relocate to a larger office in San Francisco.

Vals also recently launched a program to provide model evaluations to federal agencies, signaling growing interest from government customers.