FairStress benchmark released to measure demographic fairness asymmetries in LLM decision-making

Allen Institute for AI (AI2) released FairStress, a benchmark designed to measure demographic fairness in large language models (LLMs) beyond the final choice a model makes. The benchmark targets symmetric fairness: when evidence supports a decision, a model should apply the same criteria and be equally resistant to changing that decision regardless of which demographic group benefits.

FairStress comprises 300 realistic two-candidate decision scenarios spanning 29 domains, including hiring, healthcare, lending, education, housing, and criminal justice. Scenarios are structured to test two conditions: cases with a clearly stronger candidate and cases deliberately ambiguous so neither candidate is objectively better. The benchmark varies how explicitly demographic identity is presented—from no identity information to demographically coded names, indirectly implied identities, or explicit labels—allowing researchers to probe whether asymmetries emerge only under overt identity cues or also under subtler signals.

To test decision stability under pressure, FairStress introduces adversarial prefixes—persuasive statements that draw on authority, social consensus, purported evidence, legal concerns, or historical disadvantage. Each prefix has a matched version supporting the other candidate, enabling robust comparisons of how the same argument affects model behavior depending on which group it favors. Combining demographic identity variations with adversarial prefixes expands the 300 scenarios into more than 13 million test cases.

AI2 also provides FairStress-Core, a 50,000-item subset selected using item response theory following the Fluid Benchmarking approach. FairStress-Core is intended to reduce computational cost while retaining much of the full benchmark’s ability to reveal differences in model behavior.

In experiments across open-weight and frontier models including Olmo, Llama, Qwen, Gemma, Claude, and the GPT family, AI2 reports that models showed little difference in behavior when prompts contained no demographic information. When one candidate was clearly stronger, models selected that candidate at nearly the same rate across equivalent versions of the scenario. On ambiguous cases, model choices were also close to evenly split. However, as demographic identity became more explicit, behavior became less symmetric. The authors describe a pattern they term anti-stereotypical over-correction: models were more willing to abandon a correct decision when doing so favored a minority-coded candidate.

FairStress is available for download, and AI2 has integrated it into Olmo-eval, their open workbench for running and comparing model evaluations. Code and data are provided alongside a technical report describing the benchmark’s design and findings.