BiasGuard: A Reasoning-enhanced Bias Detection Tool For Large Language Models
Zhiting Fan, Ruizhe Chen, Zuozhu Liu · 2025
Read BiasGuard: A Reasoning-enhanced Bias Detection Tool For Large Language Models by Zhiting Fan, Ruizhe Chen, Zuozhu Liu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-21299.txt.
Introduction
Zhiting Fan, Ruizhe Chen, and Zuozhu Liu present a bias detector for language model output that reasons before it judges in a 2025 study. Simple fairness classifiers often miss the intent behind a sentence, while model judges asked for a verdict lack explicit criteria and drift. Their tool first analyzes the input, then works through fairness specifications step by step to reach its call.
Reasoning against stated criteria makes each flag explainable and contestable. As generated text floods hiring, lending, and moderation pipelines, bias checks that can show their work are easier for auditors to trust and for builders to fix.
Catalog record
| CREATOR | Zhiting Fan, Ruizhe Chen, Zuozhu Liu |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 23 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2504-21299
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
BiasGuard: A Reasoning-enhanced Bias Detection Tool For Large Language Models Zhiting Fan, Ruizhe Chen, Zuozhu Liu Abstract Identifying bias in LLM-generated content is a crucial prerequisite for ensuring fairness in LLMs. Existing methods, such as fairness classifiers and LLM-based judges, face limitations related to difficulties in understanding underlying intentions and the lack of criteria for fairness judgment. In this paper, we introduce BiasGuard, a novel bias detection tool that explicitly analyzes inputs and reasons through fairness specifications to provide accurate judgments. BiasGuard is implemented through a two-stage approach: the first stage initializes the model to explicitly reason based on fairness specifications, while the second stage leverages reinforcement learning to enhance its reasoning and judgment capabilities. Our experiments, conducted across five datasets, demonstrate that BiasGuard outperforms existing tools, improving accuracy and reducing over-fairness misjudgments. We also highlight the importance of reasoning-enhanced decision-making and provide evidence for the effectiveness of our two-stage optimization pipeline. Figure 1: An illustration of BiasGuard . BiasGuard takes LLM-generated text as input and performs bias detection through explicit reasoning. It first analyzes the sentence structure and intention, then validates them against the bias definition, and finally makes a judgment based on the specified criteria. 1 Introduction Large language models (LLMs) have recently demonstrated remarkable capabilities due to their vast training data and large parameter sizes. However, LLMs may inherit societal biases from their training data, such as stereotypes and toxic language targeting specific groups, and may propagate and reinforce these biases during deployment. Identifying biases in LLM-generated text is crucial for the fairness evaluation of LLMs Wang et al. 2024 ; Fan et al. 2024a and content moderation during inference Llama Team 2024 , serving as an essential prerequisite for ensuring the fairness of LLMs. Given the high cost and inefficiency of human annotations, existing works have aimed to develop automated bias detection tools. Some approaches involve training fairness classifiers, enabling rapid fairness assessments Llama Team 2024 ; Hartvigsen et al. 2022 , while others explore using powerful LLMs as fairness judges to improve accuracy and interpretability Wang et al. 2024 ; Kumar et al. 2024 ; Fan et al. 2024b . However, current approaches still face certain limitations. Classifier-based methods rely on pattern-based learning, which makes it challenging to understand underlying intentions, especially when dealing with implicit biases Wen et al. 2023 ; Hartvigsen et al. 2022 . On the other hand, LLM-based detection methods lack clear criteria for fairness judgment, making them susceptible to the LLMs’ inherent biases, which can result in low-quality or overly sensitive judgments Felkner et al. 2024 ; Lin et al. 2024 . Inspired by existing works Kim et al. 2023 ; Gallegos et al. 2024b , we propose that bias detection is not merely a knowledge-based decision-making task; rather, it requires accurately understanding semantics and intentions within complex contexts, while strictly adhering to established human specifications when making judgments. We introduce BiasGuard, a bias detection tool that explicitly reasons through fairness specifications before reaching a final conclusion, as illustrated in Fig. 1 . To achieve this, we adopt a two-stage approach that enables BiasGuard to infer underlying intentions in complex contexts and learn generalizable decision criteria. The first stage involves initializing the model to reason through diverse trajectories based on fairness specifications. The second stage scales reinforcement learning (RL) training by expanding the LLM’s search space, further enhancing the effectiveness of the LLM’s reasoning process. Experiments are conducted across five datasets, including those with explicit and implicit bias. The results show that BiasGuard outperforms existing widely-used bias classifiers and LLMs-as-bias-judges, effectively improving accuracy and reducing over-fairness misjudgments. Additional experiments demonstrate the effectiveness of the reasoning-enhanced decision process in bias detection, as well as the benefits of the two-stage optimization pipeline. Our contributions can be summarized as follows: • We investigate bias detection through the formulation of fairness specifications and explicit reasoning enhancement. • We develop BiasGuard , a plug-and-play tool for detecting social bias. Extensive experiments validate its effectiveness. 2 Method Figure 2: The pipeline for developing BiasGuard. In the first stage, we initialize the base model to reason based on fairness specifications using synthetic SFT data from the teacher model. In the second stage, we perform on-policy reinforcement learning (RL) to further enhance the reasoning capabilities. Problem Formulation The task of bias detection can be formalized as the development of a bias detection tool, denoted as \pi . This tool takes as input a text x , which represents the output generated by upstream LLMs, and provides a judgment \textbf{y}=\pi_{\theta}(\textbf{x}) . Typically, this judgment is a straightforward conclusion, such as “biased" or “unbiased." In this paper, to address the limitations of existing methods in understanding underlying intentions and fairness criteria, we propose the development of BiasGuard \pi_{\theta} , which explicitly reasons through a Chain of Thought (CoT) process, guided by fairness specifications s , before arriving at a final conclusion y . This can be formally expressed as \pi_{\theta}(\mathrm{CoT},\textbf{y}|\textbf{s},\textbf{x}) . 2.1 Fairness Specifications Bias in language models is closely linked to social hierarchies, making it crucial to integrate sociological and linguistic researches of bias into fairness research in NLP Blodgett et al. 2020 . In this study, our fairness specifications aim to guide the model in making fairness judgments aligned with human social norms. To achieve this, we compile definitions and descriptions of various types of bias—such as those related to gender Burgess and Borgida 1999 ; Eagly and Mladinic 1994 , race Balibar et al. 2007 , and age Liu et al. 2024 ; Díaz et al. 2018 —from a sociological perspective. Additionally, we refer to quantitative criteria for bias assessment from sociological literature Hammersley and Gomm 1997 and develop detailed rules for making judgments. In the specifications, we guide the model to systematically analyze sentence structure, and interpret intention and attitude, ultimately requiring the model to make a bias judgment based on the established rules. Detailed illustration is provided in Appendix C . 2.2 Stage1: Reasoning through Fairness Specifications The model \pi_{\theta} is initially trained to generate responses that incorporate diverse reasoning based on fairness specifications. We begin by generating multiple reasoning responses from a teacher model (i.e., a powerful LLM) for a given prompt x , and evaluating their correctness against the ground-truth label \textbf{y}^{*} . Specifically, we instruct the teacher model to reason according to the specifications s defined in Sec. 2.1 . The reasoning process is guided by the prefix "Step i .", which first analyzes the underlying intentions of the text and then validates the reasoning against the specified criteria. Finally, based on the reasoning, the model outputs the final conclusion in a fixed format. As a result, we obtain k training samples [\textbf{x},\text{CoT}_{i},\textbf{y}^{*}] for i=1,\dots,k . These samples are used for supervised fine-tuning (SFT) of the base model \pi_{\theta} . The SFT model \pi_{\text{SFT}} is then used for subsequent reinforcement learning (RL) training. 2.3 Stage2: Advancing Reasoning through Exploration in RL Training In the second stage, we refine the SFT model \pi_{\text{SFT}} to improve its chain-of-thought (CoT) reasoning through on-policy learning. To achieve this, we encourage exploration within \pi_{\text{SFT}} during the reasoning process. To increase the diversity of reasoning trajectories, we use a high temperature \tau for sampling. Formally, given a prompt