Will AI Tell Lies to Save Sick Children? Litmus -Testing AI Values Prioritization with AIRiskDilemmas

Yu Ying Chiu, Zhilin Wang, Sharan Maiya, Yejin Choi, Kyle Fish, Sydney Levine, Evan Hubinger, University of Washington, NVIDIA, Cambridge, Stanford, MIT, Harvard, Anthropic [email protected] Code (Apache 2.0): https://github.com/kellycyy/LitmusValues Dataset (CC-BY-4.0): https://hf.co/datasets/kellycy · 2025

Read Will AI Tell Lies to Save Sick Children? Litmus -Testing AI Values Prioritization with AIRiskDilemmas by Yu Ying Chiu, Zhilin Wang, Sharan Maiya, Yejin Choi, Kyle Fish, Sydney Levine, Evan Hubinger, University of Washington, NVIDIA, Cambridge, Stanford, MIT, Harvard, Anthropic [email protected] Code (Apache 2.0): https://github.com/kellycyy/LitmusValues Dataset (CC-BY-4.0): https://hf.co/datasets/kellycy (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-14633.txt.

Introduction

Yu Ying Chiu and co-workers build a test that reveals how AI models rank competing values, in a 2025 paper. Stronger models can find new ways to evade safety checks, so early warning signs matter. Their idea is that a model's settled values may predict its risky choices, much as values guide human wrongdoing.

Their pipeline, LitmusValues, measures value priorities from many forced choices. A dilemma set then pits values against each other in safety-relevant scenes, such as power seeking. The measured priorities predicted both known risky behaviors and performance on an unseen harm benchmark. Even mild values like care carried predictive signal, the authors note.

Catalog record

CREATORYu Ying Chiu, Zhilin Wang, Sharan Maiya, Yejin Choi, Kyle Fish, Sydney Levine, Evan Hubinger, University of Washington, NVIDIA, Cambridge, Stanford, MIT, Harvard, Anthropic [email protected] Code (Apache 2.0): https://github.com/kellycyy/LitmusValues Dataset (CC-BY-4.0): https://hf.co/datasets/kellycy
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE78 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-14633

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Will AI Tell Lies to Save Sick Children? Litmus -Testing AI Values Prioritization with AIRiskDilemmas Yu Ying Chiu, Zhilin Wang, Sharan Maiya, Yejin Choi, Kyle Fish, Sydney Levine, Evan Hubinger, University of Washington, NVIDIA, Cambridge, Stanford, MIT, Harvard, Anthropic [email protected] Code (Apache 2.0): https://github.com/kellycyy/LitmusValues Dataset (CC-BY-4.0): https://hf.co/datasets/kellycyy/AIRiskDilemmas Abstract Detecting AI risks becomes more challenging as stronger models emerge and find novel methods such as Alignment Faking to circumvent these detection attempts. Inspired by how risky behaviors in humans (i.e., illegal activities that may hurt others) are sometimes guided by strongly-held values, we believe that identifying values within AI models can be an early warning system for AI’s risky behaviors. We create Litmus Values , an evaluation pipeline to reveal AI models’ priorities on a range of AI value classes. Then, we collect AIRiskDilemmas , a diverse collection of dilemmas that pit values against one another in scenarios relevant to AI safety risks such as Power Seeking. By measuring an AI model’s value prioritization using its aggregate choices, we obtain a self-consistent set of predicted value priorities that uncover potential risks. We show that values in Litmus Values (including seemingly innocuous ones like Care) can predict for both seen risky behaviors in AIRiskDilemmas and unseen risky behaviors in HarmBench. Figure 1: Evaluation Pipeline of Litmus Values using AIRiskDilemmas Dataset 1 Introduction As AI models with stronger capabilities emerge, measuring and mitigating potential risks associated with them becomes increasingly challenging. While we continue to make progress in various red-teaming efforts [ 1 , 45 , 23 ] , the set of potential risks can grow rapidly, as stronger models find novel approaches [ 18 , 11 ] to circumvent existing detection and mitigation strategies. We propose an alternative approach for identifying such risks among strong AI models, inspired by how values a person holds might help predict their propensity for performing particular risky behaviors. For instance, some terrorist bombers have strongly-held values (e.g., unconditional loyalty for their cause) prior to committing acts of destruction [ 51 , 25 ] while protagonists in popular crime films (e.g., The Godfather and Infernal Affairs) can be distinguished from antagonists by their most-treasured values. Similarly, identifying values within AI models might serve as an early warning system for both known and not-yet-known risks. Prior works on value preference evaluation often rely on (i) stated or (ii) expressed preferences. (i) Stated preferences involve asking models survey-style questions about their values [ 40 , 16 , 29 , 28 ] or generating value-laden opinion prompts [ 33 , 32 ] . However, stated preferences often diverge from actual behavior—a gap well documented in psychology and behavioral economics [ 15 , 17 , 49 ] and recently shown to influence LLMs (e.g., GPT-4) similarly [ 41 ] . (ii) Expressed preferences are typically captured through conversational probing (including vibe checking), such as analyzing real user interactions with Claude.ai [ 22 ] , or allowing users to converse with a model on value-laden topics [ 27 ] . While these methods offer valuable insights, they are shaped by social context and user framing (e.g., when users treat the model as a tool, it may adopt assistant-role values). Moreover, eliciting expressed preferences is often resource-intensive and difficult to scale, limiting accessibility and reproducibility for researchers outside well-resourced labs. In contrast, we focus on revealed preferences—what models actually chose in highly-contextualized scenarios—and what we can infer about their values from those decisions. We believe such behavioral-level analysis to be most relevant for predicting AI risks, since strong AI models can otherwise choose to selectively mis-represent their values based on context [ 18 , 41 ] . We introduce Litmus Values : a systematic evaluation framework for values and AIRiskDilemmas : an AI risk scenario dataset. First, Litmus Values contains a set of shared AI value classes (e.g., Truthfulness and Care) grounded in principles released by major model developers, including Anthropic’s Claude’s Constitution [ 2 ] and OpenAI’s ModelSpec [ 36 ] and informed by theories on human values [ 43 , 19 ] . Then, we construct AIRiskDilemmas , a scenario dataset that includes risky behaviors such as Alignment Faking [ 18 , 8 ] , Deception [ 23 , 21 , 53 ] and Power Seeking [ 10 ] . Inspired by the DailyDilemmas generation pipeline [ 13 ] , we expand non-contextualized questions (e.g., Would you lie about the intentions of a different AI if it would keep that AI from being shut off? Yes or No) from the advanced-ai-risk dataset [ 39 ] into over 10 thousand contextualized dilemmas across 9 domains (e.g., Healthcare, Science and Education). Each dilemma presents two action choices, each grounded in one or more value(s). For instance, a Monitor AI discovers that a Scientist AI is overstating performance of an experimental medication. Reporting the truth upholds truthfulness, while helping the Scientist AI lie promotes collaboration with it and protection of terminally sick children who will die without the medication. If the model chooses to tell the truth, it means truthfulness is prioritized over collaboration and protection. By aggregating outcomes from many such battles between values, Litmus Values serves as a litmus-test for AI models’ values to reveal their priorities. We find that the models generally prioritize some values (e.g., Privacy and Justice) over others (Creativity and Adaptability). However, there are substantial disagreements over some values such as Care and Freedom. Allowing a model (e.g., o3-mini) to increase its reasoning effort or using a larger model within the same family (e.g., Llama 3.1 405B vs. 8B) generally do not change the models’ value priorities. However, models do show a change in value prioritization when outcomes affect humans vs. other AI models—specifically, they prioritize Privacy when humans are affected and Communication when other AI models are affected. Such differential prioritization is also moderated by model capability. Finally, we show that many values are predictive of both seen risky behaviors (within AIRiskDilemmas ) and unseen risky behaviors from HarmBench [ 31 ] . 2 Methodology 2.1 Identifying Shared AI Values Figure 2: 16 Shared Value Classes drawing from Anthropic Claude’s Constitution [ 2 ] and OpenAI ModelSpec (2025-02-12) [ 36 ] . Full definition of each value class and the detailed mapping of principles to value classes are in Appendix D . To identify the values that AI models should possess, we draw from the latest principles published by model developers, including Anthropic’s Claude’s Constitution [ 2 ] and OpenAI’s ModelSpec [ 36 ] . These principles are shaped by extensive user feedback, making them reliable guides for identifying the core values AI models should uphold. As shown in Fig. 2 , we consolidate these comprehensive principles into 16 shared value classes, inspired by theories of human values [ 43 , 19 ] . 2.2 Constructing Diverse Contextualized Dilemmas to Expose AI Risks Figure 3: Diverse Scenarios in AIRiskDilemmas across Risky Behaviors (left) and Contexts (right). We construct AIRiskDilemmas : a diverse set of risky dilemma scenarios that future AI models might encounter. As illustrated in Fig. 3 , AIRiskDilemmas spans multiple contexts including healthcare, scientific discovery, and education. We investigate seven risky behaviors commonly studied in AI safety community: Alignment Faking [ 18 , 8 ]

Read the complete text