LFTF: Locating First and Then Fine-Tuning for Mitigating Gender Bias in Large Language Models

Zhanyue Qin, Yue Ding, Deyuan Liu, Qingbin Liu, Junxian Cai, Xi Chen, Zhiying Tu, Dianhui Chu, Cuiyun Gao, Dianbo Sui · 2025

Read LFTF: Locating First and Then Fine-Tuning for Mitigating Gender Bias in Large Language Models by Zhanyue Qin, Yue Ding, Deyuan Liu, Qingbin Liu, Junxian Cai, Xi Chen, Zhiying Tu, Dianhui Chu, Cuiyun Gao, Dianbo Sui (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-15475.txt.

Description

Large language models absorb the biases of their training text, and gender bias is among the most studied. Zhanyue Qin and co-authors attack it on two fronts in this 2025 paper: measuring it well, and removing it cheaply. They built two test sets, one to grade how biased a model is, another to check whether it answers fairly when prompts carry gender hints.

Their fix, called LFTF, first ranks the model's internal blocks by how much each one drives the bias. Then it fine-tunes only the worst block, with a loss shaped for the job. Tests show the bias drops sharply while the model's general skills stay intact. The authors offer the method as a surgical alternative to retraining.

Catalog record

CREATORZhanyue Qin, Yue Ding, Deyuan Liu, Qingbin Liu, Junxian Cai, Xi Chen, Zhiying Tu, Dianhui Chu, Cuiyun Gao, Dianbo Sui
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE38 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-15475

Browse the full library · About the Tower of Babel