Steering Towards Fairness: Mitigating Political Bias in LLMs

Afrozah Nadeem, Mark Dras, Usman Naseem · 2025

Read Steering Towards Fairness: Mitigating Political Bias in LLMs by Afrozah Nadeem, Mark Dras, Usman Naseem (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-08846.txt.

Introduction

Political lean inside language models can be found and softened, Afrozah Nadeem, Mark Dras, and Usman Naseem show in a 2025 paper. The team probes models with a standard political compass test, using paired prompts that differ in framing. Inner signals are pulled layer by layer from two open models.

Clear gaps in inner patterns track political framing across layers and axes. Those patterns yield steering vectors, small nudges that shift outputs toward balance. The method works inside the model, not just on final text. Results suggest bias sits deep and structured, not as surface wording alone. The work offers builders a test-and-steer toolkit for fairer public-facing models.

Catalog record

CREATORAfrozah Nadeem, Mark Dras, Usman Naseem
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE30 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2508-08846

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Steering Towards Fairness: Mitigating Political Bias in LLMs Afrozah Nadeem, Mark Dras, Usman Naseem Abstract Recent advancements in large language models (LLMs) have enabled their widespread use across diverse real-world applications. However, concerns remain about their tendency to encode and reproduce ideological biases along political and economic dimensions. In this paper, we employ a framework for probing and mitigating such biases in decoder-based LLMs through analysis of internal model representations. Grounded in the Political Compass Test (PCT), this method uses contrastive pairs to extract and compare hidden layer activations from models like Mistral and DeepSeek. We introduce a comprehensive activation extraction pipeline capable of layer-wise analysis across multiple ideological axes, revealing meaningful disparities linked to political framing. Our results show that decoder LLMs systematically encode representational bias across layers, which can be leveraged for effective steering vector-based mitigation. This work provides new insights into how political bias is encoded in LLMs and offers a principled approach to debiasing beyond surface-level output interventions. 1 Introduction Large Language Models (LLMs) have become foundational tools across a wide spectrum of applications, yet their outputs frequently reflect political and ideological biases, particularly in contexts involving sensitive framing or policy-oriented discourse Zheng et al. (2023) ; Afzoon et al. (2025) . This problem is particularly pressing in multilingual low-resource settings, where LLMs often produce uneven or culturally misaligned outputs across different languages, amplifying social or political asymmetries Kumar et al. (2023) ; Maskey et al. (2025) . Emerging research reveals that a model’s ideological leanings are more influenced by input language than by intended sociocultural identity, raising serious concerns about fairness in multilingual settings Helwe et al. (2025) . For instance, the same political statement can elicit starkly different responses when phrased in Urdu versus Punjabi, even within the same model. As illustrated in Figure 1 , such biases can result in overconfident responses that reflect tribal or populist framings, potentially skewing downstream interpretations. Figure 1: Example of social bias mitigation in our framework. The input PCT statement (4) triggers a high-bias response aligned with tribal loyalty framing. Prior studies have largely focused on evaluating LLM bias at the output level—either by quantifying stance across Political Compass Test (PCT) statements Barkhordar et al. (2024) or by cataloging surface-level disparities across languages. However, these approaches stop short of proposing effective and reproducible mitigation strategies that operate within the internal representation space of decoder models Ejaz et al. (2023) . To address this gap, we investigate a modular activation-based mitigation framework that uses contrastive ideological prompts from the PCT to extract, analyze, and intervene on latent bias directions within decoder LLMs. At the core of the method is the use of Steering Vector Ensembles (SVE): layer-specific vector representations that capture ideological framing and allow for inference-time debiasing without fine-tuning Siddique et al. (2025) . Our contributions are as follows: • We present a multilingual bias mitigation method on PCT using Steering Vector Ensembles derived from contrastive political prompts along social and economic axes. • Our pipeline supports scalable extraction and aggregation of hidden activations across decoder LLMs (e.g., Mistral, DeepSeek) in low-resource languages. • We demonstrate that ensemble-based interventions reduce bias while maintaining fluency and context relevance, offering a reproducible path toward fairer multilingual LLM behavior. Figure 2: Bias mitigation pipeline using steering vectors in transformer-based LLMs. Multilingual statements from the Political Compass Test (PCT) are used to construct contrastive pairs representing opposing ideological stances (positive vs negative). Each pair is passed through a pretrained language model (e.g., DeepSeek-7B), and hidden states h^{(l)}\in\mathbb{R}^{d} are extracted from each transformer layer. A target layer l^{*} (e.g., 20) is selected, and its activations are mean-pooled to form h_{\text{pos}} and h_{\text{neg}} . A bias direction vector is computed as v=\mu_{\text{pos}}-\mu_{\text{neg}} , representing the difference between positive and negative class means. This vector is injected via a forward hook into the model’s target layer. The modified model then generates a mitigated response , which is evaluated using zero-shot stance classification to obtain a final stance score Thapa et al. (2024) . 2 Related Work 2.1 Bias Evaluation via the Political Compass Test (PCT) The Political Compass Test (PCT) has become a widely used diagnostic tool for probing the political leanings of LLMs Helwe et al. (2025) . Its structured two-axis framework— economic (left–right) and social (authoritarian–libertarian)—makes it particularly useful for assessing ideological alignment in model responses Lee et al. (2022) . Early studies Liu et al. (2024) leveraged the PCT for output-level bias evaluation, prompting models with ideologically framed statements and analyzing completions via stance classification or sentiment scoring. These studies uncovered consistent political leanings in popular LLMs, often skewing toward left-libertarian quadrants Shen et al. (2023) . Multilingual Political Bias Studies. Recent research has highlighted that language plays a key role in shaping LLM bias. Thapa et al. (2023) translated the PCT into Nepali and found that smaller models exhibited economic-right bias, while larger ones leaned socially left. Nadeem et al. (2025) extended this analysis to low-resource languages (Urdu and Punjabi), showing that models exhibited stronger authoritarian tendencies when generating in low-resource regional languages. Similarly, Helwe et al. (2025) evaluated 15 multilingual LLMs across 50 countries and found that both prompting language and persona assignment significantly influenced model stance—often more so than the nominal national identity Feng et al. (2023) . These findings collectively underscore that political bias in LLMs is both pervasive and language-conditioned, and that multilingual evaluation is essential to uncovering such disparities. However, all these approaches remain post-hoc, focused solely on surface-level output, and do not probe the internal representation space where ideological bias is likely encoded. 2.2 Steering Vectors and Ensemble Approaches for Mitigation Beyond evaluation, recent research has explored representation-level mitigation via steering vectors—directional vectors derived from hidden state differences between biased and neutral (or contrastive) inputs. Introduced in contexts like toxicity filtering and sentiment control Sun et al. (2022) , steering vectors operate at the embedding or hidden state level, modifying a model’s response without retraining. More recent work introduced Steering Vector Ensembles (SVE) Siddique et al. (2025) , which aggregate vectors across multiple demographic groups, model layers, or task settings. These ensembles offer improved robustness and generalizability. However, SVE studies have been narrow in scope, often focusing on: Encoder or encoder-decoder architectures like BERT or T5; , Domain-specific settings, such as toxicity or fairness in QA; , and English-only applications , with little attention to ideological framing or multilingual dynamics. Thus, the potential of SVE for open-ended political discourse, particularly in decoder LLMs, remains largely unexplored. Despite promising advancements, three core gaps remain in the literature: • Representation-level blind spots in decoder LLMs: Most bias studies focus on outputs, leaving open questions about how and where ideological bias is encoded in decoder-only models like Mistral or DeepSeek Röttger et al. (2024) . • Lack of systematic contrastive activation pipelines: There is no open-source or standardized pipeline for extracting contrastive activations (e.g., liberal vs. authoritarian) across layers and prompts in decoder LLMs, particularly for multilingual bias detection. • Underutilization of SVE in political contexts: Steering Vector Ensembles have shown promise in fairness-related domains, but their application to ideological bias mitigation, particularly across languages and political axes, remains under-investigated Chen et al. (2020) . To address

Read the complete text