Steer-Bench : A Benchmark for Evaluating the Steerability of Large Language Models
Kai Chen, Zihao He, Taiwei Shi, Kristina Lerman · 2025
Read Steer-Bench : A Benchmark for Evaluating the Steerability of Large Language Models by Kai Chen, Zihao He, Taiwei Shi, Kristina Lerman (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-20645.txt.
Introduction
Kai Chen and three co-authors build a benchmark for how well language models can adopt a community's voice, in a 2025 paper. Real users belong to groups with their own norms and styles, and a helpful model should bend toward them. That skill, called steerability, had no proper test.
Steer-Bench draws on thirty pairs of contrasting Reddit communities across nineteen domains, with more than ten thousand instruction examples and thousands of checked quiz questions. Human experts scored about eighty-one percent on the quiz labels. The best of thirteen popular models reached only about sixty-five percent, and some trailed human alignment by wide margins. The authors release the benchmark so builders can measure, and close, this gap in community-sensitive behavior.
Catalog record
| CREATOR | Kai Chen, Zihao He, Taiwei Shi, Kristina Lerman |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 74 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-20645
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Steer-Bench : A Benchmark for Evaluating the Steerability of Large Language Models Kai Chen, Zihao He, Taiwei Shi, Kristina Lerman Abstract Steerability, or the ability of large language models (LLMs) to adapt outputs to align with diverse community-specific norms, perspectives, and communication styles, is critical for real-world applications but remains under-evaluated. We introduce Steer-Bench , a benchmark for assessing population-specific steering using contrasting Reddit communities. Covering 30 contrasting subreddit pairs across 19 domains, Steer-Bench includes over 10,000 instruction-response pairs and validated 5,500 multiple-choice question with corresponding silver labels to test alignment with diverse community norms. Our evaluation of 13 popular LLMs using Steer-Bench reveals that while human experts achieve an accuracy of 81% with silver labels, the best-performing models reach only around 65% accuracy depending on the domain and configuration. Some models lag behind human-level alignment by over 15 percentage points, highlighting significant gaps in community-sensitive steerability. Steer-Bench is a benchmark to systematically assess how effectively LLMs understand community-specific instructions, their resilience to adversarial steering attempts, and their ability to accurately represent diverse cultural and ideological perspectives. 1 1 1 Code and data are available at https://github.com/kaichen23/steer-bench . 1 Introduction Figure 1: Illustration of LLM steering using community-specific instruction-response pairs. A vanilla LLM selects answer B to a multiple-choice question. After being steered using data from either subreddit r/linux or r/windows , the same model shifts its output to reflect the targeted community perspective, selecting answer A or C respectively. This demonstrates how open-ended demonstrations can guide models to adopt distinct community viewpoints. Large language models (LLMs) have demonstrated remarkable capabilities in following instructions, generating coherent text, and adapting to various contexts Lou et al. (2024) ; Tam et al. (2024) ; Zhang et al. (2023) . A key dimension of these capabilities is steerability —the ability of an LLM to tailor its outputs in response to specific guidance, constraints, or norms provided by users or developers. Among these capabilities, steerability has emerged as an important dimension for practical applications. Steerability refers to a large language model’s ability to adapt its outputs according to specific guidance, preferences, norms, or constraints provided by users or developers. Figure 1 illustrates steering LLMs using instruction-response pairs. Steerability encompasses multiple related behaviors. At its most basic, it includes instruction following : generating outputs that conform to explicit prompts. More advanced forms include model steering , where outputs are conditioned on explicit attributes or requirements; role-playing , in which models adopt consistent personas or communication styles Shanahan et al. (2023) ; Wang et al. (2024a) ; and personalization , where outputs reflect individual user preferences Li et al. (2024b) ; Kumar et al. (2025) . Another essential behavior for many real-world applications is norm following —i.e., adhering to the values, rules, and discourse patterns of particular communities Shi et al. (2024b) ; Liu et al. (2025) ; Li et al. (2024a) . While numerous benchmarks exist to evaluate general LLM capabilities, such as reasoning, reading comprehension Rajpurkar et al. (2016) ; Dua et al. (2019) , instruction following Ouyang et al. (2022) , ethical alignment Hendrycks et al. () , and personalization Kumar et al. (2025) , few are explicitly designed to evaluate higher-order capabilities like steerability. This limits our ability to systematically measure model alignment with diverse social, cultural, or ideological contexts. To address this gap, we introduce Steer-Bench , a benchmark designed to evaluate population-specific steering in LLMs. Steer-Bench draws on Reddit, a platform comprising thousands of topic-specific communities, many of which represent contrasting perspectives on shared issues (e.g., r/liberals vs. r/conservatives , r/parenting vs. r/Childfree , r/Linux vs. Windows ). These forums provide natural discourse that reflects distinct norms, rhetorical styles, and worldviews of distinct communities, making Reddit an ideal setting for assessing whether models can produce outputs that are not only coherent but also contextually appropriate across different social or cultural contexts. Steer-Bench includes 30 contrasting subreddit pairs across 19 domains, from which we curated over 10,000 open-ended question-answer pairs that reflect diverse community viewpoints. The benchmark also features a set of multiple-choice questions with corresponding silver labels, validated by human annotators, to provide high-quality ground truth for alignment evaluation. We use Steer-Bench to evaluate the steerability of 13 popular open-source and proprietary LLMs. We show that steerability improves with model size in both in-context learning and instruction-tuning settings. Furthermore, we find that contextualized instructions significantly enhance steering performance relative to relying solely on pretrained knowledge. Finally, we demonstrate that model families exhibit distinct sensitivities to steering across domains, underscoring the importance of training methodology and architecture in enabling subpopulation alignment. Figure 2: Construction of Steer-Bench with an illustrative example of the Parenting domain. First, comments from contrasting subreddits r/Parenting and r/Childfree are processed through topic modeling to identify shared discussion topics. Then, for each topic, relevant comments from each subreddit are sampled and prompt GPT-4o to generate open-ended instruction-response pairs and multiple-choice questions-answer pairs, reflecting community-specific perspectives. Our key contributions are: • We introduce STEER-BENCH, a benchmark for evaluating community-specific steerability in LLMs across 30 subreddit pairs and 19 domains, covering over 10,000 instruction-response pairs and 5,500 multiple-choice questions grounded in contrasting online communities. • We evaluate 13 popular LLMs under both in-context learning and supervised finetuning settings, revealing that steerability improves with model scale and contextualized prompting, but varies significantly by model family and domain. • We identify substantial performance gaps between models and human-aligned labels—especially in ideologically sensitive domains—highlighting the challenges of aligning LLMs with diverse cultural and social norms. 2 Related Work Instruction following Instruction following has become foundational for large language models, encompassing techniques like instruction tuning Ouyang et al. (2022) ; Wang et al. (2023) and in-context learning Zhao et al. (2025) ; Edwards and Camacho-Collados (2024) . Recent research has proposed comprehensive benchmarks and frameworks to systematically evaluate the proficiency of LLMs in executing instructed tasks Zeng et al. (2024) ; Zhou et al. (2023) . These benchmarks critically assess approaches aimed at improving LLMs’ ability to interpret and adhere to complex instructions involving multiple constraints He et al. (2024a) ; Sun et al. (2024) ; Zhang et al. (2024) . Additionally, Tam et al. (2024) investigates the impact of structured output constraints on LLM performance, providing further insights into factors influencing instruction-following effectiveness. Model steering Researchers have explored various methods for steering the behavior of large language models, seeking to enhance utility, controllability Bayat et al. (2025) ; Guo et al. (2024) , and alignment with human preferences Alves et al. (2023) . Dong et al. (2023) introduce SteerLM, which conditions responses on explicit attributes during inference, providing a supervised fine-tuning alternative to RLHF for direct behavioral control. Similarly, Cao et al. (2024) creates effective steering vectors through bi-directional preference optimization that precisely influence generation probabilities based on human preference pairs. However, challenges remain: Chen et al. (2024a) reveals LLMs’ susceptibility to ideological steering through minimal biased data exposure, Santurkar et al. (2023) finds that attempts to steer LLMs toward specific demographic viewpoints through prompting resulted in only modest improvements, indicating difficulties in effectively aligning models with diverse human opinions. Our work focuses on systematically measuring the steering capabilities of large language models. Role-play The role-playing capabilities of LLMs have emerged as a