Interpretable Debiasing of Vision-Language Models for Social Fairness

Na Min An, Yoonna Jang, Yusuke Hirota, Ryo Hachiuma, Isabelle Augenstein, Hyunjung Shim · 2026

Read Interpretable Debiasing of Vision-Language Models for Social Fairness by Na Min An, Yoonna Jang, Yusuke Hirota, Ryo Hachiuma, Isabelle Augenstein, Hyunjung Shim (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-24014.txt.

Description

Na Min An and five co-authors introduce DeBiasLens, an interpretable, model-agnostic framework for reducing social bias in vision-language models. The 2026 paper argues that current debiasing mostly attacks surface bias signals after the fact, leaving the model's internal dynamics unexplored and its reasoning opaque.

DeBiasLens trains sparse autoencoders on multimodal encoders, using face images or captions without social attribute labels, to find neurons highly responsive to specific demographics, including underrepresented groups. Selectively deactivating the neurons most tied to bias for each group reduces biased behavior without degrading semantic knowledge. The work lays groundwork for auditing tools that prioritize social fairness in real-world AI systems, by making bias localizable inside the model rather than merely measurable in its outputs. In short, the fix works inside the model, not just on its outputs.

Catalog record

CREATORNa Min An, Yoonna Jang, Yusuke Hirota, Ryo Hachiuma, Isabelle Augenstein, Hyunjung Shim
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE63 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602-24014

Browse the full library · About the Tower of Babel