Mind the Inclusivity Gap: Multilingual Gender-Neutral Translation Evaluation with mGeNTE

Beatrice Savoldi, Giuseppe Attanasio, Eleonora Cupin, Eleni Gkovedarou , Janiça Hackenbuchner , Anne Lauscher , Matteo Negri , Andrea Piergentili , Manjinder Thind , Luisa Bentivogli Fondazione Bruno Kessler, Instituto de Telecomunicações, DIT Forlì - University of Bologna, Data Science Group - Univ · 2025

Read Mind the Inclusivity Gap: Multilingual Gender-Neutral Translation Evaluation with mGeNTE by Beatrice Savoldi, Giuseppe Attanasio, Eleonora Cupin, Eleni Gkovedarou , Janiça Hackenbuchner , Anne Lauscher , Matteo Negri , Andrea Piergentili , Manjinder Thind , Luisa Bentivogli Fondazione Bruno Kessler, Instituto de Telecomunicações, DIT Forlì - University of Bologna, Data Science Group - Univ (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2501-09409.txt.

Description

Beatrice Savoldi and co-authors test whether language models can translate without forcing a gender the source never gave. Many tongues mark gender on near every noun, so a lazy model defaults to male forms. Their 2025 study builds a hand-checked test set and runs top instruction models on four language pairs.

Models could tell when neutral forms fit the source. They just could not make those forms with any steadiness. That gap caps their use in fair public text. Deeper probes show which inner marks steer the choice. The team frames neutral translation as a skill models name but do not yet hold. Their set gives the field a shared test.

Catalog record

CREATORBeatrice Savoldi, Giuseppe Attanasio, Eleonora Cupin, Eleni Gkovedarou , Janiça Hackenbuchner , Anne Lauscher , Matteo Negri , Andrea Piergentili , Manjinder Thind , Luisa Bentivogli Fondazione Bruno Kessler, Instituto de Telecomunicações, DIT Forlì - University of Bologna, Data Science Group - Univ
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE66 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2501-09409

Browse the full library · About the Tower of Babel