NeuronTune: Towards Self-Guided Spurious Bias Mitigation
Guangtao Zheng, Wenqian Ye, Aidong Zhang · 2025
Read NeuronTune: Towards Self-Guided Spurious Bias Mitigation by Guangtao Zheng, Wenqian Ye, Aidong Zhang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-24048.txt.
Introduction
Guangtao Zheng, Wenqian Ye, and Aidong Zhang built a self guided fix for spurious bias in deep networks. Their paper, from twenty twenty-five, deals with a known flaw: models lean on side cues that oft ride with a class, like a backdrop, not the thing it self. Such models fail when the cue is gone. Past fixes need hand marks of which cues are false, which are dear to get and may miss the model's own traps.
Their method, NeuronTune, works after training. It probes the model's inner space to find the neurons that drive the false picks, then tunes them down. Proof backs the path. Readers get a fix that asks no outside marks of bias.
Catalog record
| CREATOR | Guangtao Zheng, Wenqian Ye, Aidong Zhang |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 92 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-24048
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
NeuronTune: Towards Self-Guided Spurious Bias Mitigation Guangtao Zheng, Wenqian Ye, Aidong Zhang Abstract Deep neural networks often develop spurious bias, reliance on correlations between non-essential features and classes for predictions. For example, a model may identify objects based on frequently co-occurring backgrounds rather than intrinsic features, resulting in degraded performance on data lacking these correlations. Existing mitigation approaches typically depend on external annotations of spurious correlations, which may be difficult to obtain and are not relevant to the spurious bias in a model. In this paper, we take a step towards self-guided mitigation of spurious bias by proposing NeuronTune, a post hoc method that directly intervenes in a model’s internal decision process. Our method probes in a model’s latent embedding space to identify and regulate neurons that lead to spurious prediction behaviors. We theoretically justify our approach and show that it brings the model closer to an unbiased one. Unlike previous methods, NeuronTune operates without requiring spurious correlation annotations, making it a practical and effective tool for improving model robustness. Experiments across different architectures and data modalities demonstrate that our method significantly mitigates spurious bias in a self-guided way. 1 Introduction Deep neural networks trained using empirical risk minimization (ERM) often develop spurious bias: a tendency to rely on spurious correlations for predictions. A spurious correlation refers to a non-causal relationship between a class and an attribute that is not essential for defining the class, commonly referred to as a spurious attribute ( Ye et al., 2024 ) . For example, the class of waterbird and the background of the water can form spurious correlations in the predictions of waterbird ( Sagawa et al., 2019 ) , as the background of the water is a spurious attribute. In contrast, core attributes, such as bird feathers, causally determine a class. A model with spurious bias may achieve a high prediction accuracy ( Beery et al., 2018 ; Geirhos et al., 2019 ; Geirhos et al., 2020 ; Xiao et al., 2021 ; Zheng et al., 2024a ) even without core attributes, such as identifying an object only by its frequently co-occurring background ( Geirhos et al., 2020 ) . However, the model may perform poorly on the data lacking the learned spurious correlations, which poses a great challenge to robust model generalization. Existing methods ( Sagawa et al., 2019 ; Kirichenko et al., 2023 ; Deng et al., 2024 ) that mitigate spurious bias are mostly at the sample level, using a curated set of samples with annotations of spurious correlations called group labels to retrain a biased model. A group label (class, spurious attribute) annotates a sample with a spurious attribute and its class label, representing a spurious correlation. However, group labels are difficult to acquire and often require costly human-guided annotations. To circumvent this, group label estimation ( Nam et al., 2022 ) and various sample reweighting mechanisms ( Nam et al., 2020 ; Liu et al., 2021 ; Kim et al., 2022 ; Qiu et al., 2023 ; LaBonte et al., 2024 ) are adopted using the idea that spurious bias can be identified through the misclassification of bias-conflicting samples. Despite significant progress in spurious bias mitigation, existing sample-level methods that rely on group labels or sample reweighting offer limited and indirect control over how spurious bias is addressed. On the one hand, group labels are data annotations that are external to a model and may not accurately reflect the specific spurious bias developed in the model. On the other hand, sample reweighting does not directly target the internal mechanisms that give rise to spurious bias. This highlights the need for a self-guided approach that directly intervenes in a model’s decision process , providing more targeted and model-relevant signals for mitigating spurious bias than sample-level approaches. To this end, we focus on developing self-guided methods that directly analyze the internal prediction mechanism of a model to identify components of the model that are affected by spurious bias and then mitigate their influence to final predictions. We take a step towards this goal by proposing a novel method termed NeuronTune , which systematically reduces spurious bias in deep neural networks. NeuronTune first probes in the latent embedding space of a trained model to identify dimensions (neurons) of sample embeddings affected by spurious bias, termed biased dimensions —those where spurious attributes predominantly contribute to prediction errors ( Bykov et al., 2023 ; Singla & Feizi, 2021 ) . Those dimensions can be identified when high activation magnitudes are strongly associated with incorrect predictions, indicating that features represented by those dimensions are not truly predictive of target classes. Importantly, rather than attempting to explicitly distinguish dimensions representing spurious and core attributes, an inherently challenging task given the complex entanglement of features in deep networks, NeuronTune instead identifies biased dimensions and suppresses the contributions of the these dimensions to final predictions. This intervention encourages the model to discover robust decision rules and mitigates spurious bias in the model. Compared with the existing sample-level methods for spurious bias mitigation, NeuronTune provides direct intervention at the neuron level, allowing for more precise and targeted control over the mitigation of spurious bias during model tuning. Unlike approaches that rely on sample-level annotations such as group labels, NeuronTune enables the model to self-debias without external supervision. This makes it applicable in standard ERM training settings, where no additional annotations beyond class labels are available. As a result, NeuronTune serves as a practical and effective post hoc tool for mitigating spurious bias. We theoretically demonstrate that neuron activations coupled with their final prediction outcomes provide self-identifying information on whether the neurons are affected by spurious bias. Our theoretical findings further suggest a practical metric for identifying biased dimensions and proves that NeuronTune can bring a model closer to the unbiased one. Experiments on vision and text datasets with different model architectures confirm the effectiveness of our method. 2 Related Work Depending on the availability of external supervision, we summarize prior spurious bias mitigation methods into supervised, semi-supervised, and unsupervised categories. Supervised Spurious Bias Mitigation. In this setting, certain spurious correlations in data are given in the form of group labels. With group labels in the training data, balancing the size of the groups ( Cui et al., 2019 ; He & Garcia, 2009 ) , upweighting groups that do not have specified spurious correlations ( Byrd & Lipton, 2019 ) , or optimizing the worst-group objective ( Sagawa et al., 2019 ) can be effective. Regularization strategies, such as using information bottleneck ( Tartaglione et al., 2021 ) or the distributional distance between bias-aligned samples ( Barbano et al., 2023 ) , are also proved to be effective in spurious bias mitigation. The concept of neural collapse has also been exploited recently for spurious bias mitigation ( Wang et al., 2024 ) . However, this setting requires to know what spurious bias needs to be mitigated a priori and only focuses on mitigating the specified spurious bias. Semi-Supervised Spurious Bias Mitigation. This setting aims to mitigate spurious bias without extensive spurious correlation annotations. A small portion of group labels in a held-out set are required for achieving optimal performance. One line of works is to use data