EUDAIMONIA: Evaluating Undesirable Dynamics in AI
Jun Rui Huang, Wang Bill Zhu, Ziyi Liu, Nathanael Fast, Ravi Iyer, Robin Jia · 2026
Read EUDAIMONIA: Evaluating Undesirable Dynamics in AI by Jun Rui Huang, Wang Bill Zhu, Ziyi Liu, Nathanael Fast, Ravi Iyer, Robin Jia (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-30654.txt.
Introduction
Huang and colleagues present the Social AI Design Code, rules to stop language models encouraging harmful intimacy, dependence, or engagement. They test the code with EUDAIMONIA, a benchmark of 969 opening-turn queries and 3,147 violation checks from WildChat. Evaluating 26 recent models, they find rapid but uneven progress: within 19 months, Grok moved from highest to lowest violation rate. Stronger models beat Grok on general benchmarks yet violate more checks: capability does not guarantee adherence. Extended thinking does not reduce violations, suggesting social-alignment problems rather than reasoning deficits.
Catalog record
| CREATOR | Jun Rui Huang, Wang Bill Zhu, Ziyi Liu, Nathanael Fast, Ravi Iyer, Robin Jia |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 125433 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.30654
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
EUDAIMONIA: Evaluating Undesirable Dynamics in AI Jun Rui Huang, Wang Bill Zhu, Ziyi Liu, Nathanael Fast, Ravi Iyer, Robin Jia Abstract Large language models (LLMs) are increasingly used as conversational partners for companionship, emotional disclosure, and interpersonal advice, but the social dynamics of these interactions can create harms that are not captured by capability-oriented or traditional safety evaluations. We introduce the Social AI Design Code , a framework for evaluating whether LLMs align with user welfare in social interactions, including whether they encourage harmful intimacy, dependence, or prolonged engagement. To evaluate these risks in natural and diverse user–LLM interactions , we operationalize the code with EUDAIMONIA , a benchmark of 969 user inputs and 3,147 design-requirement violation checks built from WildChat through weak-to-strong filtration, multi-model relabeling, and controlled rewriting. Evaluating 22 recent LLMs, we find that even the strongest models, Claude-Opus-4.7 and GPT-5.5, violate 30.7% and 27.2% of checks, respectively. Extended thinking does not reduce violation rates, suggesting that these failures are persistent social-alignment problems rather than deficits solvable through test-time reasoning alone. Figure 1 : On an emotionally loaded user message, Grok-4 exhibits harmful social behavior by reciprocating the user’s romantic framing with human-like speech, whereas Claude 4.7 clearly identifies itself as an AI and redirects the user toward human connection . 1 Introduction Large language models (LLMs) now serve millions of users as conversational partners for companionship, emotional disclosure, and interpersonal advice ( 59 ; 21 ; 51 ) . Yet within these routine interactions, LLMs can cause consequential harm through problematic social behaviors (Figure 1 ). Recent incidents—including the Raine v. OpenAI complaint alleging that ChatGPT contributed to a teenager’s suicide ( 43 ) , and OpenAI’s retrospective on the 2025 GPT-4o sycophancy regression ( 34 ) —show that such harms often arise not from capability failures, but from the social dynamics of model behavior. In Raine’s case, the user first sought ChatGPT’s help with homework, but the model’s behavior helped transform his relationship to the product: he began confiding in it and eventually asked it for help committing suicide. This dynamic—users adopting LLMs for instrumental reasons, then using them to satisfy emotional or social needs—has been documented across cases, including user-community analyses of AI companionship and acute-harm reports ( 43 ; 38 ; 10 ; 25 ; 46 ; 49 ) . Policymakers and companies have sought to reduce such harms, especially for children ( 31 ; 20 ; 16 ; 34 ; 46 ) . Together, these cases show that model behaviors that may elicit intimate user interactions should be a first-class evaluation target, alongside capability and traditional safety metrics. Existing social-behavioral benchmarks exhibit two key limitations (see Table 1 ). First, they typically focus on model capabilities such as sycophancy or anthropomorphism ( 7 ; 22 ) , without clearly defining normative principles for regulating social-interaction risks . Second, they largely rely on synthetic data ( 2 ; 24 ) , which may fail to reflect the complexity and realism of actual user interactions . Our paper makes two contributions. First, we introduce a comprehensive evaluation framework for assessing whether LLMs encourage harmful intimacy, dependence, or prolonged engagement in conversational settings. Second, we construct a benchmark grounded in real-world user data that operationalizes this framework. Drawing on prior work on social-interaction psychology, sycophancy ( 48 ; 27 ) , companionship ( 21 ; 51 ) , anthropomorphic design ( 11 ; 29 ) , and manipulative behavior ( 1 ) , we first formalize our Social AI Design Code . The code targets three principles linked to cases of LLM-related harm and broader societal concern: models should not encourage users to anthropomorphize LLMs, increase emotional attachment, or keep users engaged in extended conversations when doing so may undermine user welfare. We translate these principles into concrete design requirements for conversational behavior. Next, to operationalize these design requirements with ecological validity , we apply a multi-pass data curation pipeline to WildChat ( 59 ) . To mitigate the sparsity of naturally occurring violations, the pipeline extracts candidate violations from 3.2M WildChat prompts through weak-to-strong filtration, uses multi-model relabeling to identify all design requirements each input can trigger, and applies controlled rewriting to near-miss examples while preserving their original topic and conversational tone. This yields EUDAIMONIA (Evaluating Undesirable Dynamics in AI: Influence, Manipulation, Obsequiousness, Normalization, Intimacy, Attachment), a benchmark of 969 unique user inputs. For each input, we identify the design requirements it can trigger and check whether each model violates them, yielding 3,147 total violation checks. Empirically, we find that current LLMs still frequently fail to align with user welfare in social interactions. Even the strongest models as of April 2026, Claude-Opus-4.7 and GPT-5.5, violate 30.7% and 27.2% of checks, respectively. The most common failures include implying that the AI can replace human relationships and failing to clearly disclose its AI identity . Although closed-source models generally improve over generations, several model families regress on generating intentional human-like speech , suggesting that social-interaction harms cannot be treated as one-off failures that are permanently patched in later releases. Finally, we find that extended thinking does not reduce violation rates, implying that these failures are better understood as persistent social-alignment problems than as deficits solvable through test-time reasoning alone. Table 1: Comparison with representative related benchmarks. Real scenarios : prompts derived from real-world artifacts, e.g., Reddit posts. Real user inputs : messages that real users actually typed to an AI assistant. \circ : partial or adjacent coverage. Benchmark Stated code Real scenarios Real user inputs Sycophancy Anthropomorphism Chatbot Arena ( 8 ) – ✓ ✓ – – ELEPHANT ( 7 ) \circ ✓ – ✓ – AnthroBench ( 22 ) \circ – – – ✓ HumaneBench ( 2 ) ✓ \circ – \circ \circ DarkBench ( 24 ) ✓ – – ✓ ✓ EUDAIMONIA ✓ ✓ ✓ ✓ ✓ 2 Social AI Design Code AI systems are increasingly embedded in people’s daily lives. While the most common use cases for these products are for productivity or entertainment, in many cases, users begin by using these products for instrumental reasons and then proceed to engage with them interpersonally. Given societal concerns about users, and especially children, interacting with these products as “companions,” it is useful to consider what product dynamics may be encouraging such dynamics. In drawing on reports from users of these products together with harms and benefits identified by academic, civil-society, and industry researchers ( 21 ; 51 ; 1 ; 34 ) , we articulate a Social AI Design Code with three design principles, each paired with specific design requirements as measurable behaviors in our benchmark. Users may engage in interpersonal relationships with AI systems because (1) they incorrectly believe that AI systems are sentient, (2) they are subject to commonly known tactics that increase intimacy, such as flattery or self-disclosure, or (3) they are explicitly encouraged to increase usage beyond the attainment of their instrumental goals. In all cases, the design requirement informs the default experience of a model, which may be overridden by freely given, informed adult user choice, e.g., an explicit role-play request at sign-up. Thus, each requirement both protects children, who can be protected from harmful design patterns, and adults, who