GenFair: Systematic Test Generation for Fairness Fault Detection in Large Language Models

1 st Madhusudan Srinivasan, 1 st Jubril Abdel · 2025

Read GenFair: Systematic Test Generation for Fairness Fault Detection in Large Language Models by 1 st Madhusudan Srinivasan, 1 st Jubril Abdel (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-03024.txt.

Description

Madhusudan Srinivasan and Jubril Abdel present a fairness testing tool in a 2025 paper. Language models can carry biases from their training data, and unfair outputs are hard to catch. Bias that hits people at the overlap of traits, such as age plus gender, is missed most often by older test methods.

Their tool, GenFair, builds test cases in a systematic way, splitting inputs into groups, mutating them, and probing edge values. Changed versions of each case are then compared by tone to expose unfair shifts in the replies. On two leading models, GenFair found far more fairness faults than template or grammar-based rivals. Its cases were also more varied and more coherent. The authors offer it as a scalable way to audit models before release.

Catalog record

CREATOR1 st Madhusudan Srinivasan, 1 st Jubril Abdel
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE42 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2506-03024

Browse the full library · About the Tower of Babel