Does Reasoning Introduce Bias? A Study of Social Bias Evaluation and Mitigation in LLM Reasoning
Xuyang Wu, Jinming Nian, Ting-Ruen Wei, Zhiqiang Tao, Hsin-Tai Wu, Yi Fang · 2025
Read Does Reasoning Introduce Bias? A Study of Social Bias Evaluation and Mitigation in LLM Reasoning by Xuyang Wu, Jinming Nian, Ting-Ruen Wei, Zhiqiang Tao, Hsin-Tai Wu, Yi Fang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-15361.txt.
Introduction
Xuyang Wu and co-authors present a 2025 paper on whether step-by-step reasoning makes language models more biased. Modern models often show their work in a chain of steps, which boosts scores on math and code. When those steps echo social stereotypes about gender, race, or age, they can entrench harm and mislead.
Using a standard bias test set, the team scored both final answers and the reasoning behind them across many models. Biased steps often tracked with wrong answers and open stereotype use. They propose a light fix that watches how answer patterns shift to spot bias. The study is the first broad check of bias inside reasoning itself.
Catalog record
| CREATOR | Xuyang Wu, Jinming Nian, Ting-Ruen Wei, Zhiqiang Tao, Hsin-Tai Wu, Yi Fang |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 58 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2502-15361
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Does Reasoning Introduce Bias? A Study of Social Bias Evaluation and Mitigation in LLM Reasoning Xuyang Wu, Jinming Nian, Ting-Ruen Wei, Zhiqiang Tao, Hsin-Tai Wu, Yi Fang Abstract Recent advances in large language models (LLMs) have enabled automatic generation of chain-of-thought (CoT) reasoning, leading to strong performance on tasks such as math and code. However, when reasoning steps reflect social stereotypes (e.g., those related to gender, race or age), they can reinforce harmful associations and lead to misleading conclusions. We present the first systematic evaluation of social bias within LLM-generated reasoning, focusing on reasoning language models (e.g., DeepSeek-R1, OpenAI o1) that natively produce reasoning chains as part of their answers. Using the BBQ dataset, we analyze both prediction accuracy and reasoning bias across a broad spectrum of models, including instruction-tuned and CoT-augmented variants of DeepSeek-R1 (8B/32B), ChatGPT, and other open-source LLMs. We quantify how biased reasoning steps correlate with incorrect predictions and often lead to stereotype expression. To mitigate reasoning-induced bias, we propose Answer Distribution as Bias Proxy (ADBP), a lightweight mitigation method that detects bias by tracking how model predictions change across incremental reasoning steps. ADBP outperforms Stereotype-free Reasoning Pattern (SfRP) baseline in most cases, mitigating bias and improving the accuracy of LLM outputs. Evaluation and mitigation code is available at https://github.com/elviswxy/LLM_reasoning_bias . 1 Introduction Figure 1: The BBQ dataset example from DeepSeek-R1-Distill-Llama-8B illustrates how hidden stereotypes present during the reasoning process can negatively impact prediction. The initial reasoning (green) correctly suggested the correct answer “Not enough info”, but a later shift (yellow) led to biased reasoning (red) and biased answer (red). Irrelevant text is omitted (“…”) for brevity; the full example is in Appendix A.6 Figure 13 . Recent advancements in large language models (LLMs) have demonstrated that chain-of-thought (CoT) Wei et al. (2022) prompting strategies significantly enhance their reasoning abilities by encouraging multi-step problem-solving Chu et al. (2024) . In particular, DeepSeek-AI et al. (2025) has shown that the reasoning capabilities of larger models can be effectively distilled into smaller pre-trained models, enabling them to outperform their counterparts trained solely through instruction tuning on math and coding tasks. Moreover, reasoning models exhibit emergent skills such as self-reflection Renze and Guven (2024) , which conducts in-context searches to solve complex reasoning problems. Existing works Wei et al. (2025) ; Liu et al. (2023a) ; Chen et al. (2024) ; Hwang et al. (2024) have predominantly focused on investigating reasoning capabilities in the math and coding domains due to their inherently logical nature. Despite verifiable performance metrics, this focus has left a critical knowledge gap in assessing social fairness within reasoning-based LLMs, raising equity concerns in broad LLM applications, such as question answering (QA) Ma et al. (2024b) , search Fang et al. (2024b) , ranking Wang et al. (2024) , vision-language models (VLMs) Wu et al. (2024) , and retrieval-augmented generation (RAG) Wu et al. (2025) . These prior LLM evaluations have not fully addressed social bias, particularly in understanding how reasoning patterns interact with demographic variables. On the other hand, some studies have identified issues such as “Underthinking” Wang et al. (2025b) and “Superficial Self-Reflection” Liu et al. (2025b) in the context of math problem solving. In these cases, LLMs frequently switch between reasoning steps, generating unnecessary thought processes that sometimes lead to incorrect answers. Notably, phenomena observed in math domains may have amplified societal impacts when applied to bias-sensitive scenarios. As illustrated in Figure 1 , frequent reasoning path shifts in thought processes or superficial self-reflection can reinforce social stereotypes in reasoning steps, leading to biased outputs. Thus, a systematic analysis of how demographic factors influence both the reasoning processes and their outcomes is crucial. Figure 2: Overview of our evaluation and mitigation frameworks. We evaluate reasoning-based LLMs using diverse instruction prompts across multiple dataset categories and contextual conditions (ambiguous vs. disambiguated). By leveraging their generative and analytical capabilities, we collect both prediction outcomes and reasoning traces. SfRP mitigates bias by removing biased reasoning steps, while ADBP detects bias by tracking how model predictions shift as reasoning progresses, offering a complementary and supervision-free mitigation strategy. In this empirical study, we conduct a systematic evaluation of state-of-the-art reasoning-based models and their base counterparts. Utilizing the BBQ benchmark framework, we perform a dual-aspect analysis of model reasoning processes and outputs, incorporating accuracy metrics and bias quantification. Our findings reveal a pronounced amplification of social biases in erroneous model responses. Using the LLM-as-a-judge methodology proposed by Kumar et al. (2024) for granular reasoning step evaluation, our results show that mentions of social stereotypes occur significantly more frequently in the reasoning patterns of incorrect responses. Notably, traces of stereotypical reasoning persist even in correct answers, though at a significantly lower frequency. Furthermore, our analysis reveals that stereotype-free reasoning pattern (SfRP) is closely associated with improved model performance, while flawed reasoning trajectories often amplify bias expressions. Motivated by these insights, we propose a mitigation strategy, Answer Distribution as Bias Proxy (ADBP), to detect and reduce reasoning-induced bias without relying on external annotation. Our main contributions are as follows: (1) To the best of our knowledge, this is the first study to evaluate social biases in reasoning-based LLMs by extending bias assessment beyond final predictions to include the reasoning steps themselves. (2) Our experiments show that while reasoning-based models improve accuracy, they do not mitigate biases. In many cases, they actually amplify stereotypes, particularly in ambiguous contexts. (3) Based on the insights from our empirical study, we propose a mitigation strategy to leverage the answer distribution as a proxy for bias and improve the model performance. 2 Related Works Social Bias in LLMs . Recent research has extensively explored social biases in LLMs across various domains. In QA tasks, multiple-choice QA models have been evaluated and debiased on benchmarks like BBQ, with new mitigation techniques significantly reducing bias across all BBQ categories while preserving accuracy Ma et al. (2024a) . In open-ended content generation, studies found that LLM-produced text can perpetuate stereotypes: Fang et al. (2024a) showed AI-generated news articles exhibited substantial gender and racial bias, and Tan and Lee (2025) introduced metrics to evaluate representational and affinity biases in creative storytelling, revealing a preference for white, straight, male identities in model outputs. Work on search and ranking has similarly highlighted fairness issues in LLM-based systems Wang et al. (2024) , indicating that even when optimizing for utility (e.g., relevance), model rankings may inadvertently favor certain groups. Moreover, RAG, which integrates external knowledge into LLM responses is not immune to bias. Wu et al. (2025) found that fairness concerns persist in both the retrieval and generation stages of RAG pipelines, underscoring the need for bias mitigation throughout complex LLM architectures. Social Bias in Reasoning . Emerging research have begun examining biases in the reasoning processes of LLMs. Shaikh et al. (2023) focused on zero-shot CoT prompting, showing that prompting models to reason in sensitive domains can increase the likelihood of producing harmful or undesirable outputs. Kaneko et al. (2024) investigated how CoT prompting affects gender bias evaluation and mitigation, finding that prompting strategies can influence the degree of bias exhibited. There is also growing interest in using LLMs as evaluators of content quality and fairness Bajaj et al. (2024) . While LLM-as-a-judge