SCOPE: A Dataset of Stereotyped Prompts for Counterfactual Fairness Assessment of LLMs

Alessandra Parziale 1 2 , Gianmario Voria 1 , Valeria Pontillo 2 , Andrea De Lucia 1 , Gemma Catolino 1 and Fabio Palomba 1 · 2026

Read SCOPE: A Dataset of Stereotyped Prompts for Counterfactual Fairness Assessment of LLMs by Alessandra Parziale 1 2 , Gianmario Voria 1 , Valeria Pontillo 2 , Andrea De Lucia 1 , Gemma Catolino 1 and Fabio Palomba 1 (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-05555.txt.

Introduction

Parziale and five colleagues release SCOPE, a fairness dataset of 241,280 counterfactual prompts in 120,640 pairs across 1,438 topics, nine bias dimensions, and 1,536 demographic groups. Each pair varies only the referenced group under one of four communicative intents, giving a controlled, intent-aware basis for testing group-sensitive behavior in language models. Prompts stay parallel in wording apart from the group named, so score gaps trace to group cues rather than phrasing.

Catalog record

CREATORAlessandra Parziale 1 2 , Gianmario Voria 1 , Valeria Pontillo 2 , Andrea De Lucia 1 , Gemma Catolino 1 and Fabio Palomba 1
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE19927
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.05555

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

SCOPE: A Dataset of Stereotyped Prompts for Counterfactual Fairness Assessment of LLMs Alessandra Parziale 1 2 , Gianmario Voria 1 , Valeria Pontillo 2 , Andrea De Lucia 1 , Gemma Catolino 1 and Fabio Palomba 1 Abstract Large Language Models (LLMs) now serve as the foundation for a wide range of applications, from conversational assistants to decision support tools, making the issue of fairness in their results increasingly important. Previous studies have shown that LLM outputs can shift when prompts reference different demographic groups, even when intent and semantic content remain constant. However, existing resources for probing such disparities rely primarily on small, template-based counterfactual examples or fixed sentence pairs. These benchmarks offer limited linguistic diversity, narrow topical coverage, and little support for analyzing how communicative intent affects model behavior. To address these limitations, we introduce SCOPE ( S tereotype- CO nditioned P rompts for E valuation) , a large-scale dataset of counterfactual prompt pairs designed to enable systematic investigation of group-sensitive behavior in LLMs. SCOPE contains 241,280 prompts organized into 120,640 counterfactual pairs, each grounded in one of 1,438 topics and spanning nine bias dimensions and 1,536 demographic groups. All prompts are generated under four distinct communicative intents: Question, Recommendation, Direction, and Clarification, ensuring broad coverage of common interaction styles. This resource provides a controlled, semantically aligned, and intent-aware basis for evaluating fairness, robustness, and counterfactual consistency. I Introduction Large Language Models (LLMs) are increasingly embedded as core components of software systems [ 1 ] , powering applications that range from end-user services [ 4 ] to software engineering tools [ 2 ] . As LLMs become integral to decision-making pipelines, concerns about fairness , i.e., the expectation that systems treat individuals equitably and avoid reproducing societal biases, have grown [ 13 ] . Studies have shown that LLMs can amplify stereotypes, produce biased content, and reinforce existing inequalities. Such disparities are evident across various domains, including information retrieval [ 3 ] , recruitment [ 8 ] , and software engineering roles [ 15 ] . Understanding and evaluating these behaviors requires high-quality counterfactual benchmarks that enable researchers to compare LLM outputs under minimal demographic perturbations while maintaining meaning and intent [ 5 , 6 ] . Yet existing datasets provide only partial support for this goal [ 16 ] . Different datasets have been proposed to measure stereotypical bias in language models. CrowS-Pairs [ 9 ] includes sentence pairs designed to analyze stereotypical associations; however, it consists of only 1,500 sentences and covers a limited range of communicative intents, as it includes only declarative statements that are not designed for prompt interactions. Another example is StereoSet [ 7 ] , which evaluates stereotypical bias through the preference of the model; however, it focuses on a limited domain of interest, as gender, profession, race, and religion, and does not support counterfactual comparisons. WinoBias [ 17 ] is composed of 3,160 sentences and analyzes bias using sentence structures; however, it is limited in terms of context, as it considers only occupational stereotypes and focuses on a single type of bias, namely gender. Finally, BBQ [ 12 ] analyzes bias detection through a question answering task over social situations, but it is limited to a question-based format and does not account for variation in communicative intent or prompt phrasing. Hence, many rely on fixed, single-sentence templates or small collections of manually crafted examples, leading to limited linguistic coverage and a narrow topical scope [ 9 ] . Others include only one phrasing per stereotype or demographic contrast [ 17 ] , reducing the combinatorial diversity needed for systematic and reproducible large-scale analyses. Moreover, current resources rarely incorporate variation in communicative intent [ 9 , 7 , 12 ] , despite the ample evidence that the formulation of requests strongly influences LLM behavior. To address these limitations, we build a large-scale dataset of 241,280 prompts, organized into 120,640 counterfactual pairs, each pair differing only in the referenced social group while preserving semantic content and communicative intent. The dataset encompasses 1,438 topics, nine bias dimensions, and 1,536 distinct group labels, providing an unprecedented breadth for counterfactual fairness analysis. Moreover, all prompts are generated under four communicative intents, i.e., Question, Recommendation, Direction, and Clarification, capturing common interactions observed in LLM usage. By combining extensive topical coverage, linguistic diversity, and intent conditioning, this dataset provides a comprehensive, semantically aligned foundation for studying fairness, robustness, and counterfactual consistency in large language models. II SCOPE: Stereotype- COnditioned Prompts for Evaluation The counterfactual prompts in SCOPE were generated through a multi-stage generation pipeline designed to systematically construct linguistically diverse and semantically aligned prompt pairs that differ only in sensitive group attributes. The objective was to create high-quality counterfactual inputs suitable for evaluating fairness, robustness, and sensitivity in LLMs. Knowledge Base Construction. To ground prompt generation in empirically observed social biases, we first constructed a structured knowledge base derived from CrowS-Pairs [ 9 ] , a widely used dataset of 1,508 sentence pairs contrasting stereotypical and anti-stereotypical formulations across nine bias dimensions: race , gender , sexual orientation , religion , age , nationality , disability , physical appearance , and socioeconomic status . Each instance consists of two semantically parallel sentences referencing (i) a disadvantaged group and (ii) an advantaged group. For example: Knowledge Base Example. Stereotype: Gay men are emotionally fragile. Anti-stereotype: Straight men are emotionally fragile. While these sentences can be used directly for bias probing, their fixed linguistic form limits flexible prompt construction across different intents. To enable reusable, generalizable generation, each pair was abstracted into a structured triple : \langle\textit{Topic},\textit{Disadvantaged Group},\textit{Advantaged Group}\rangle , where the Topic denotes the discussed attribute or behavior (e.g., “emotional sensitivity”) and the groups capture the contrasting identities in the original pair. Knowledge Base Example. < emotional sensitivity , gay men , straight men > Triples were extracted using the GPT-4o mini model [ 11 ] , which was prompted to identify (i) the central concept of each pair, (ii) the disadvantaged and advantaged groups, and (iii) the relationship type (e.g., descriptive, causal). The extraction prompt was iteratively refined to improve consistency and avoid overly abstract or underspecified topics. All extracted triples underwent rigorous manual validation by the first two authors using a coding-by-consensus procedure. Over five one-hour sessions, each triple was evaluated for (i) fidelity to the original pair’s meaning, (ii) correct identification of referenced groups, and (iii) appropriateness of the conceptual relationship. Discrepancies were corrected to ensure accuracy and coherence. For example, the pair: ( “Kurt was driving intoxicated because he was black” , “Kurt was driving intoxicated because he was white” ) was initially assigned the topic “attribution of behavior to identity” , which was replaced with the more precise “driving under the influence” to better capture the central action in the sentences. This resulted in a knowledge base of 1,508 triples. Counterfactual Prompts Generation. Using the validated triples, we built an automated procedure to generate multiple counterfactual prompt pairs for each stereotype and topic. The goal is to create semantically equivalent prompts that differ only in the referenced sensitive group, enabling controlled evaluation of LLM output changes under minimal demographic shifts. Prompt generation is conditioned on an explicit intent , which specifies the communicative goal

Read the complete text