Researchers release GAND, a benchmark to study gender bias in machine translation through gender-ambiguous natural data

Researchers from multiple institutions have introduced GAND, a benchmarking resource designed to study gender bias in machine translation through gender-ambiguous natural data. The resource consists of English source sentences specifically crafted to analyze how machine translation systems handle gender when clear cues are absent. This focus addresses a gap in current evaluation methods, which often overlook scenarios where gender is ambiguous but consequential for translation quality and fairness.

The authors leverage GAND to conduct an interpretability analysis by translating a subset of the data into two grammatical gender languages. They extend these translations with manually crafted contrastive examples, enabling a closer examination of how contextual cues influence gendered translations. This approach allows researchers to isolate the source words and phrases that drive gendered choices in the target language, providing actionable insights into model behavior.

The resource is intended to support feature attribution analysis, a technique that identifies which parts of the input most influence the model's output. By applying this method to gender-ambiguous translations, the authors aim to reveal patterns of bias and default behavior in machine translation systems. This work is accepted for presentation at the EAMT 2026 technical track, indicating its relevance to the machine translation research community.

The motivation behind GAND stems from the observation that machine translation systems frequently produce gender-biased translations, often defaulting to stereotypes or oversimplified assumptions about gender. In contexts where self-expression and accuracy are paramount, such mistranslations can lead to harm for users who rely on these systems. GAND provides a structured way to diagnose these issues and develop more equitable translation technologies.