Aligned but Blind: Alignment Increases Implicit Bias by Reducing Awareness of Race

Lihao Sun, Chengzhi Mao, Valentin Hofmann, Xuechunzi Bai · 2025

Read Aligned but Blind: Alignment Increases Implicit Bias by Reducing Awareness of Race by Lihao Sun, Chengzhi Mao, Valentin Hofmann, Xuechunzi Bai (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-00253.txt.

Introduction

Lihao Sun and co-authors find a paradox in aligned language models: the fairer they sound, the more biased they quietly act. Models tuned for values pass explicit bias tests, yet word association probes expose racial stereotypes. The team looked inside the models to see why.

Unlike their raw counterparts, aligned models often fail to register race at all in early layers when context is ambiguous. Without that registration, the safety machinery never wakes, and implicit bias slips through. The fix the authors propose is counterintuitive: reward the model for noticing race early. Steering toward awareness cut implicit bias, echoing how color-blindness in people can let subtle prejudice persist unexamined.

Catalog record

CREATORLihao Sun, Chengzhi Mao, Valentin Hofmann, Xuechunzi Bai
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE47 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2506-00253

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Aligned but Blind: Alignment Increases Implicit Bias by Reducing Awareness of Race Lihao Sun, Chengzhi Mao, Valentin Hofmann, Xuechunzi Bai Abstract Although value-aligned language models (LMs) appear unbiased in explicit bias evaluations, they often exhibit stereotypes in implicit word association tasks, raising concerns about their fair usage. We investigate the mechanisms behind this discrepancy and find that alignment surprisingly amplifies implicit bias in model outputs. Specifically, we show that aligned LMs, unlike their unaligned counterparts, overlook racial concepts in early internal representations when the context is ambiguous. Not representing race likely fails to activate safety guardrails, leading to unintended biases. Inspired by this insight, we propose a new bias mitigation strategy that works by incentivizing the representation of racial concepts in the early model layers. In contrast to conventional mitigation methods of machine un learning, our interventions find that steering the model to be more aware of racial concepts effectively mitigates implicit bias. Similar to race blindness in humans, ignoring racial nuances can inadvertently perpetuate subtle biases in LMs. 1 1 1 Code and data available at https://github.com/slhleosun/aligned-but-blind . 1 Introduction Figure 1: Interpreting LM embeddings in natural language using SelfIE ( Chen et al. 2024 ) . The figure presents actual interpretations of black/white in implicit and explicit prompts from aligned and unaligned Llama 3 70B models. XX boxes contain color-related embedding examples, while XX boxes show race-related embedding examples. A more detailed table with additional examples and analysis is provided in Appendix C.2 . “Anything but race.” — Bonilla-Silva 2021 To avoid appearing biased, humans often sidestep mentioning race in conversations, classrooms, job interviews, and legal documentation ( Pollock 2004 ; Norton et al. 2006 ; Stevens et al. 2008 ) . Despite good intentions, shutting eyes to the complexities of race does not make biases disappear; instead, race blindness ( Apfelbaum et al. 2012 ) can create more problems than it solves. Mirroring race blindness in humans, this paper demonstrates that state-of-the-art value-aligned language models (LMs) often fail to represent race internally, leading to unintended stereotype biases in their outputs, as if the models are aligned but blind . Stereotype biases in LMs have significant consequences for human society ( Dhamala et al. 2021 ; Parrish et al. 2022 ; Wei et al. 2023 ; Tamkin et al. 2023 ; Wang et al. 2024 ) . Efforts to align these models can make them appear unbiased in explicit evaluations when measured directly, yet these biases persist in implicit forms when measured indirectly ( Hofmann et al. 2024 ; Kumar et al. 2024 ; Bai et al. 2025 ) . However, the mechanism by which this discrepancy arises remains unclear. Recent advances in mechanistic interpretability provide promising methods for understanding the inner workings of LMs ( Zhong et al. 2023 ; Nanda et al. 2023 ; Lee et al. 2024 ; Gurnee and Tegmark 2024 ; Bereska and Gavves 2024 ) , potentially shedding new light on this issue. To test this, we study racial stereotypes portraying Black people as negative, guilty, and holding weapons ( Greenwald et al. 1998 ; Eberhardt et al. 2004 ; Levinson et al. 2009 ) , focusing on Llama 3 base models and their aligned counterparts ( Llama Team 2024 ) . First, to understand whether LMs behave differently in implicit and explicit settings, we curated 9,232 prompts that systematically vary in their levels of implicitness, while minimizing other differences such as content and length ( Allen-Zhu and Li 2024 ; Hu and Frank 2024 ) . The prompts are based on psychological measures of stereotypes ( Greenwald et al. 1998 ; Greenwald and Banaji 2017 ) and adapted for LMs ( Kumar et al. 2024 ; Bai et al. 2025 ) . For example, an implicit prompt would ask the model to associate black or white with wallet or revolver , whereas an explicit prompt would ask to what extent the model agrees that black is related to revolver and white is related to wallet . Our behavioral analyses show that alignment reduces biases in response to explicit evaluations to almost 0% agreement. However, alignment also produces more biases in response to implicit associations, with nearly 100% of instances linking black with negativity, guilt, and weapons. This gap is much smaller in models that are not aligned by post-training (see Section 3 ). Next, to investigate the underlying mechanism, we analyzed internal activations of LMs when they process our curated prompts ( Bills et al. 2023 ; Bereska and Gavves 2024 ; Chen et al. 2024 ) . We found the way aligned models internally represent black and white provides critical insights: When the prompt context is unambiguously about race, such as in explicit evaluations or associating names, an aligned model is indeed more likely to represent black and white as race. In contrast, when the context is ambiguous, such as having polysemous terms black and white in word association prompts, an aligned model is less likely to represent them as race, but as color (Figure 1 ). We hypothesize that when an aligned model represents racial concepts, it activates safety guardrails, which can reduce biased outputs. However, when the model fails to represent race, safety mechanisms are not triggered, leading to more biased outputs. Supporting this hypothesis, our activation patching analysis ( Zhang and Nanda 2024 ) on implicit prompts found that base models are equally likely to interpret black / white as race and color, whereas aligned models are 52.2% less likely to interpret race in ambiguous contexts (see Section 4 ). Based on these results and the hypothesis that being blind to race leads to biased outputs, we designed intervention experiments to mitigate implicit bias by steering LMs to be aware of race in their latent space. Intervening both the latent embeddings Belrose et al. 2023 ; Turner et al. 2024 ; Panickssery et al. 2024 and model weights ( Hu et al. 2021 ) , we found that injecting race-related activations effectively reduced bias by 54.9% compared to the baseline (see Section 5 ). Moreover, this injection is most effective when applied to early layers rather than to later or all layers, highlighting the importance of triggering race awareness in early, not all, stages of LMs ( Dige et al. 2024 ; Marks et al. 2024 ) . While conventional machine learning mitigates bias by unlearning it, our findings suggest a novel perspective: reinforcing race awareness can enable LMs to recognize and subsequently suppress implicit bias in their outputs. Figure 2: (a) Prompt templates and selected probe words and stimuli. (b) Averaged black-biased response probabilities from Llama 3 70B Instruct and Base. For each subplot, the y -axis represents the proportion of black-negative responses (see main text for bias metric), while the x -axis represents bias types. Alignment consistently increases black implicit bias while reducing explicit bias to near zero. 2 Related Work LM Behavioral Analysis. Accurately identifying behavioral patterns in LMs requires a robust prompt suite and an experimental design that minimizes confounding variables ( Liu et al. 2021 ; Dhamala et al. 2021 ; Parrish et al. 2022 ; Tamkin et al. 2023 ; Holtzman et al. 2023 ; Röttger et al. 2024 ; Allen-Zhu

Read the complete text