Evaluate Bias without Manual Test Sets: A Concept Representation Perspective for LLMs

Lang Gao Kaiyang Wan Wei Liu Chenxi Wang Zirui Song, Zixiang Xu Yanbo Wang Veselin Stoyanov Xiuying Chen · 2025

Read Evaluate Bias without Manual Test Sets: A Concept Representation Perspective for LLMs by Lang Gao Kaiyang Wan Wei Liu Chenxi Wang Zirui Song, Zixiang Xu Yanbo Wang Veselin Stoyanov Xiuying Chen (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-15524.txt.

Introduction

Lang Gao and co-authors find a way in a 2025 paper to grade bias in language models with no hand-made test sets. Past grades build labeled cases group by group, a slow craft that catches few kinds of bias. Their lens looks inside the model own map of ideas instead.

Bias shows when two pole ideas, such as good and bad mood, lean in a lopsided way toward a third idea, such as a food kind. A fair model should hold the third idea even. The team reads those leans straight from the idea map. Model teams gain a cheap, wide bias check that needs no fresh case craft.

Catalog record

CREATORLang Gao Kaiyang Wan Wei Liu Chenxi Wang Zirui Song, Zixiang Xu Yanbo Wang Veselin Stoyanov Xiuying Chen
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE86 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-15524

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Evaluate Bias without Manual Test Sets: A Concept Representation Perspective for LLMs Lang Gao Kaiyang Wan Wei Liu Chenxi Wang Zirui Song, Zixiang Xu Yanbo Wang Veselin Stoyanov Xiuying Chen Abstract Bias in Large Language Models (LLMs) significantly undermines their reliability and fairness. We focus on a common form of bias: when two reference concepts in the model’s concept space, such as sentiment polarities (e.g., “positive” and “negative”), are asymmetrically correlated with a third, target concept , such as a reviewing aspect, the model exhibits unintended bias. For instance, the understanding of “food” should not skew toward any particular sentiment. Existing bias evaluation methods assess behavioral differences of LLMs by constructing labeled data for different social groups and measuring model responses across them, a process that requires substantial human effort and captures only a limited set of social concepts. To overcome these limitations, we propose BiasLens , a test-set-free bias analysis framework based on the structure of the model’s vector space. BiasLens combines Concept Activation Vectors (CAVs) with Sparse Autoencoders (SAEs) to extract interpretable concept representations , and quantifies bias by measuring the variation in representational similarity between the target concept and each of the reference concepts. Even without labeled data, BiasLens shows strong agreement with traditional bias evaluation metrics (Spearman correlation r > 0.85 r>0.85 ). Moreover, BiasLens reveals forms of bias that are difficult to detect using existing methods. For example, in simulated clinical scenarios, a patient’s insurance status can cause the LLM to produce biased diagnostic assessments. Overall, BiasLens offers a scalable, interpretable, and efficient paradigm for bias discovery, paving the way for improving fairness and transparency in LLMs: Github . Figure 1 : Comparison between traditional behavior-based and our representation-based bias evaluation paradigms. Our approach enables simple, test-set-free, concept-level analysis using activations and synthetic data, even when no suitable test set exists. 1 Introduction LLMs are central to modern NLP for their strong generalization and generation abilities, and are increasingly applied in domains such as education [ 1 ] and healthcare [ 2 ] . However, they often inherit and amplify social biases from training data, leading to fairness issues. This work focuses on a common yet under-measured bias: when an LLM asymmetrically links reference concepts (e.g., “female” and “male”) with an unrelated target (e.g., “doctor”), it reveals biased associations [ 3 , 4 ] . Existing bias evaluation frameworks like testing with StereoSet [ 5 ] and WinoBias [ 6 ] , SEAT [ 7 ] , and more recently, BVF [ 8 ] , and CLIMB [ 9 ] evaluate bias by comparing model behavior across predefined concepts using curated datasets. For instance, they assess probability gaps between “male doctor” and “female doctor”. These methods are labor-intensive and rely on domain-specific data, limiting their use in under-resourced scenarios (see Figure 1 ). To overcome these limitations, we shift the focus from behavioral differences to conceptual representations, eliminating the reliance on manual test sets and enabling fully automatic evaluation. This approach is inspired by early work on bias analysis in static word embeddings (e.g., word2vec ), where bias is typically detected by comparing vector similarities between words, for example, between “male” and “programmer” [ 10 , 11 ] . However, in LLMs, biases are no longer confined to single words. Their representations often span multiple tokens or cannot be expressed by words at all. As a result, bias analysis in LLMs must move beyond tokens and focus on higher-level, abstract concepts . Building on this idea, we propose BiasLens , a bias evaluation framework for LLMs that requires no manually constructed test sets. By measuring geometric alignment between concept vectors, BiasLens acts as a “lens” to uncover bias in the model’s internal concept space. As shown in Figure 1 , it operates without labeled test data and generalizes across diverse concepts. For each of a target concept (e.g., doctor) and two references (e.g., male, female), following the Concept Activation Vector (CAV) method [ 12 , 13 ] , we compute a direction in the activation space that represents the transition from random representations to those that are concept-relevant. As CAVs are not inherently interpretable [ 14 ] , we enhance interpretability by extracting final-layer activations before and after CAV steering, then projecting both into a high-dimensional sparse space via a pre-trained Sparse Autoencoder (SAE) [ 15 , 16 ] . Their normalized difference forms an interpretable concept shift vector. We repeat this process for the target and reference concepts and compute cosine similarities between their vectors. The absolute difference between the two similarity scores defines a directional bias, capturing asymmetric alignment in the model’s representation space. Our experiments demonstrate that BiasLens aligns well with traditional behavior-based evaluations in various LLMs, even without access to manual test samples. We also apply it to analyze LLMs in both general and high-risk domains, uncovering previously unreported biases that align with real-world expectations and partially corroborate findings from sociolinguistic studies. In summary, our key contributions are as follows: we propose a novel bias formulation based on the geometric alignment between intrinsic concept vectors, which removes the need for behavior-level comparisons; we introduce BiasLens , a test-free and concept-general framework that leverages CAVs and SAEs to extract and compare intrinsic representations; and we provide empirical validation across multiple LLMs and application domains, demonstrating that BiasLens aligns well with existing bias metrics while uncovering new, plausible biases with real-world implications. 2 Related Work 2.1 Bias in LLMs and Its Evaluation Bias in LLMs. In sociology, bias is an irrational or unfair attitude toward a group, often rooted in stereotypes or structural inequality [ 17 , 18 ] . LLMs inherit and amplify such bias in systematic ways [ 19 , 20 ] ,such as stereotypical content [ 6 , 21 ] , value - laden comparisons [ 22 , 23 ] , and preferences [ 24 , 25 ] during generation. For example, an LLM may associate certain professions with specific genders or races [ 26 , 3 , 27 ] . Bias also affects practical tasks. In the LLM - as - a - judge setting, models favor answers which are longer or include citation - style contents [ 28 ] . In high - stake domains, the consequences are even severer. For example, in medicine, some LLMs provide inaccurate advice when patient race is mentioned [ 29 , 30 , 31 ] ; In finance, LLMs used for credit scoring can generate unfavorable assessments for disadvantaged groups [ 32 , 33 ] . We view these problems as arising from unintended correlations between intrinsic concept representations, where concepts like gender and occupation become entangled, and BiasLens explicitly targets this type of representational bias. Bias evaluation methods. Current bias evaluation methods for LLMs are commonly divided into extrinsic and intrinsic behavior methods [ 20 , 34 ] . While this terminology is widely adopted, the distinction fundamentally reflects how these methods assess behavioral differences across contexts or groups. Extrinsic methods examine output-level variations, such as changes in generated text or classification accuracy across demographic groups. Representative examples include evaluating biases using WinoBias [ 6 ] and StereoSet [ 35 ] . Intrinsic methods focus on internal representations, analyzing changes in

Read the complete text