LFTF: Locating First and Then Fine-Tuning for Mitigating Gender Bias in Large Language Models
Zhanyue Qin, Yue Ding, Deyuan Liu, Qingbin Liu, Junxian Cai, Xi Chen, Zhiying Tu, Dianhui Chu, Cuiyun Gao, Dianbo Sui · 2025
Read LFTF: Locating First and Then Fine-Tuning for Mitigating Gender Bias in Large Language Models by Zhanyue Qin, Yue Ding, Deyuan Liu, Qingbin Liu, Junxian Cai, Xi Chen, Zhiying Tu, Dianhui Chu, Cuiyun Gao, Dianbo Sui (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-15475.txt.
Description
Large language models absorb the biases of their training text, and gender bias is among the most studied. Zhanyue Qin and co-authors attack it on two fronts in this 2025 paper: measuring it well, and removing it cheaply. They built two test sets, one to grade how biased a model is, another to check whether it answers fairly when prompts carry gender hints.
Their fix, called LFTF, first ranks the model's internal blocks by how much each one drives the bias. Then it fine-tunes only the worst block, with a loss shaped for the job. Tests show the bias drops sharply while the model's general skills stay intact. The authors offer the method as a surgical alternative to retraining.
Catalog record
| CREATOR | Zhanyue Qin, Yue Ding, Deyuan Liu, Qingbin Liu, Junxian Cai, Xi Chen, Zhiying Tu, Dianhui Chu, Cuiyun Gao, Dianbo Sui |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 38 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-15475