BiasEdit: A Training-Free Bias-Detect-and-Edit Framework for Learning Fair Visual Classifiers
Jungwook Seo, Yoonsik Park, Changmin Lee, Sungyong Baik † · 2026
Read BiasEdit: A Training-Free Bias-Detect-and-Edit Framework for Learning Fair Visual Classifiers by Jungwook Seo, Yoonsik Park, Changmin Lee, Sungyong Baik † (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-28450.txt.
Introduction
Jungwook Seo and three colleagues present BiasEdit, a training-free framework that finds and edits bias attributes in web image data. It first detects unknown bias attributes through statistical dependence in vision-language representations, then edits them with text-guided image editing to create realistic counterexamples. No manual annotations or known bias labels are needed, and off-the-shelf models suffice. Even on fully biased training data, the method reaches state-of-the-art debiasing for visual classifiers.
Catalog record
| CREATOR | Jungwook Seo, Yoonsik Park, Changmin Lee, Sungyong Baik † |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 58179 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.28450
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
BiasEdit: A Training-Free Bias-Detect-and-Edit Framework for Learning Fair Visual Classifiers Conference: Proceedings of the ACM Web Conference 2026; April 13–17, 2026; Dubai, United Arab Emirates. Proceedings of the ACM Web Conference 2026 (WWW ’26), April 13–17, 2026, Dubai, United Arab Emirates ISBN: 979-8-4007-2307-0/2026/04 DOI: 10.1145/3774904.3792411 Thanks: † Corresponding author. CCS: Information systems Web applications CCS: Computing methodologies Computer vision Jungwook Seo, Yoonsik Park, Changmin Lee, Sungyong Baik † Abstract . Visual data from the Web power image classifiers, which underpin web services, including recommendation and moderation. However, Web data contain spurious correlations and social biases, and neural networks tend to learn biases in data. This can reinforce unfairness in web services and the web data, leading to a vicious cycle. In image classification, networks learn bias attributes for a specific class when most images contain the same attribute for a given class. Hence, training a fair and debiased classifier from a biased dataset demands handling imbalance between a majority of images with bias attributes (bias-aligned samples) and a minority without (bias-conflict samples). In this work, we introduce BiasEdit, a modular framework that automatically detects bias attributes from the original dataset and edits them to construct a debiased dataset. Specifically, BiasEdit first detects unknown bias attributes via statistical dependence and mutual information analysis of visual–linguistic representations, and then explicitly edits those attributes using text-guided image editing to generate realistic bias-conflict samples. Unlike prior works that assume known bias attributes or rely on synthetic mixing, our method operates without manual annotations and leverages off-the-shelf vision–language and editing models. BiasEdit addresses a fundamental challenge in Web-sourced visual AI, mitigating dataset-induced bias and achieving state-of-the-art debiasing performance even when training data are fully biased. † † cc-license: by 1. Introduction The Web has become the largest and most diverse source of visual data for training modern AI systems ( 38 ; 6 ) . From social media platforms and e-commerce sites to search and content moderation pipelines, Web-sourced visual data power classifiers that interpret and organize online content ( 12 ; 14 ; 5 ; 44 ; 45 ; 20 ) . However, such data inevitably reflect the social and contextual biases present in human behavior—for example, correlations between gender and occupation or between background and object category. Neural networks trained on these datasets tend to exploit these spurious correlations, resulting in biased classifiers, undermining its fidelity and fairness ( 1 ; 9 ) . When deployed in Web services, these models can reinforce stereotypes, distort recommendations, or unfairly filter content, amplifying existing biases at scale. Figure 1 . Illustration of the BiasEdit pipeline. BiasEdit first detects unknown biases. Then, detected biases are used to create bias-conflict samples via editing of bias and target attributes, such that dataset biases will be mitigated. One of the causes for the problem is the tendency of neural networks to learn to exploit bias attributes that exist in datasets ( 10 ) . A bias attribute refers to an attribute that frequently appears in a given dataset and forms a spurious correlation (i.e., non-causal misleading relationship) with the target attribute. For example, if a large number of ‘pedestrian’ images in a dataset contain ‘crosswalk’ background, a spurious correlation is present between a bias attribute (‘crosswalk’) and a target attribute (‘pedestrian’). The abundance of bias attributes in a biased dataset makes it easier for neural networks to minimize empirical risks by learning the spurious correlations between bias and target attributes, a phenomenon known as shortcut learning ( 9 ) . Debiasing aims to train models to focus on target features—essential characteristics that define each class—while ignoring spurious correlations in biased datasets. Early approaches ( 37 ; 21 ; 41 ; 35 ) assume that all bias attributes are known, resulting in methods tailored to these biases. Such strong assumption requires extensive attribute annotation, limiting scalability and generalizability to unknown or overlooked biases. Some works leverage prior bias knowledge ( 42 ; 2 ) , but access to such information is scarce in real-world settings. More recently, research has advanced debiasing without the known-bias assumption ( 33 ; 26 ; 34 ; 18 ) . These methods typically identify bias-conflict samples—based on the notion that bias attributes are learned faster than target attributes—and upweight them during training. However, they are limited by the small number of bias-conflict samples, which poses a risk of overfitting. To overcome overfitting, strategies for generating bias-conflict samples have been explored, including generative models ( 11 ; 17 ; 22 ) , mixup augmentation ( 47 ; 16 ) , or adversarial examples ( 29 ) . Nevertheless, these generated samples are often unrealistic and retain substantial bias, as most methods do not explicitly detect or manipulate bias attributes but use indirect approaches like sample mixing or minor pixel perturbations. In this work, by contrast, we aim to create realistic bias-conflict samples. To this end, we propose a framework that performs three steps: (1) detecting bias attributes; (2) editing both bias attributes and target attributes in order to produce realistic bias-conflict samples; and (3) train a debias ed classifier on newly constructed datasets, as illustrated in Figure 1 . To detect bias attributes, we first extract attributes from images using a vision-language model ( 15 ; 30 ) . Then, we propose a statistical bias attribute detection to identify bias attributes from extracted attributes by using statistical relationships across classes, specifically statistical dependence and mutual information. In order to gain fine control over the editing of bias and target attribute in newly created images, we employ text-guided editing ( 31 ; 4 ; 48 ) to explicitly edit bias and target attributes in images, thereby producing bias-conflict samples. Experimental results demonstrate that our proposed approach, dubbed BiasEdit , not only creates realistic bias-conflict samples but also outperforms existing baselines. Notably, BiasEdit achieves robust performance even in extreme scenarios, such as when the dataset contains up to 100\% bias-aligned samples. 2. Related works The growing reliance on Web-sourced visual data has amplified long-standing concerns about bias in machine learning. Web-collected datasets inevitably contain social and contextual imbalances that reflect human behavior and online content patterns. These biases propagate through downstream classifiers, affecting fairness in Web services such as content moderation, search, and recommendation. Consequently, debiasing techniques have become a critical component of responsible Web AI. Early debiasing approaches assume the full knowledge of bias attributes, relying on meticulously labeled datasets or known bias types ( 37 ; 21 ; 41 ; 35 ; 10 ; 42 ; 28 ; 2 ) . This assumption, however, limits their applicability in real-world scenarios where such detailed annotation is costly and rarely available. To address this limitation, recent studies tackle unknown-bias scenarios. Many leverage the observation that models learn bias attributes faster than target attributes, using loss from an auxiliary biased model to identify and up-weight bias-conflict samples ( 33 ; 26 ; 18 ) . Others utilize attribute representations ( 25 ; 27 ) or analyze relationships between bias attributes and network depth ( 43 ; 40 ) . We note that a fundamental challenge is the severe imbalance: bias-conflict samples are often scarce. To overcome this, some works generate bias-conflict samples using simple augmentations or manipulations, such as