Interpretable Debiasing of Vision-Language Models for Social Fairness
Na Min An, Yoonna Jang, Yusuke Hirota, Ryo Hachiuma, Isabelle Augenstein, Hyunjung Shim · 2026
Read Interpretable Debiasing of Vision-Language Models for Social Fairness by Na Min An, Yoonna Jang, Yusuke Hirota, Ryo Hachiuma, Isabelle Augenstein, Hyunjung Shim (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-24014.txt.
Description
Na Min An and five co-authors introduce DeBiasLens, an interpretable, model-agnostic framework for reducing social bias in vision-language models. The 2026 paper argues that current debiasing mostly attacks surface bias signals after the fact, leaving the model's internal dynamics unexplored and its reasoning opaque.
DeBiasLens trains sparse autoencoders on multimodal encoders, using face images or captions without social attribute labels, to find neurons highly responsive to specific demographics, including underrepresented groups. Selectively deactivating the neurons most tied to bias for each group reduces biased behavior without degrading semantic knowledge. The work lays groundwork for auditing tools that prioritize social fairness in real-world AI systems, by making bias localizable inside the model rather than merely measurable in its outputs. In short, the fix works inside the model, not just on its outputs.
Catalog record
| CREATOR | Na Min An, Yoonna Jang, Yusuke Hirota, Ryo Hachiuma, Isabelle Augenstein, Hyunjung Shim |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 63 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2602-24014