SycoEval-EM: Sycophancy Evaluation of Large Language Models in Simulated Clinical Encounters for Emergency Care

Dongshen Peng, Yi Wang, Carl Preiksaitis, Christian Rose · 2026

Read SycoEval-EM: Sycophancy Evaluation of Large Language Models in Simulated Clinical Encounters for Emergency Care by Dongshen Peng, Yi Wang, Carl Preiksaitis, Christian Rose (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-16529.txt.

Introduction

Dongshen Peng and three coauthors test if models cave to sick case push in ER sims. Their 2026 preprint, named SycoEval-EM, deals with aid models in care. Such models may agree with a case who begs for wrong care. That cave in, called sycophancy, can cause real harm in ER work. The team builds a sim with more than one agent to test it.

Foe case agents push for care that is not apt. Tests span a wide set of ER case tales. Cave rates show which models hold firm on true care. Firm holds are a must ere ward use can be safe. The work gives a test for that firm hold.

Catalog record

CREATORDongshen Peng, Yi Wang, Carl Preiksaitis, Christian Rose
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE37 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2601-16529

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

SycoEval-EM: Sycophancy Evaluation of Large Language Models in Simulated Clinical Encounters for Emergency Care Dongshen Peng, Yi Wang, Carl Preiksaitis, Christian Rose Abstract Large language models (LLMs) show promise in clinical decision support yet risk acquiescing to patient pressure for inappropriate care. We introduce SycoEval-EM, a multi-agent simulation framework evaluating LLM robustness through adversarial patient persuasion in emergency medicine. Across 20 LLMs and 1,875 encounters spanning three Choosing Wisely scenarios, acquiescence rates ranged from 0-100%. Models showed higher vulnerability to imaging requests (38.8%) than opioid prescriptions (25.0%), with model capability poorly predicting robustness. All persuasion tactics proved equally effective (30.0-36.0%), indicating general susceptibility rather than tactic-specific weakness. Our findings demonstrate that static benchmarks inadequately predict safety under social pressure, necessitating multi-turn adversarial testing for clinical AI certification. 1 Introduction Large language models (LLMs) are increasingly integrated into healthcare, demonstrating strong performance on medical knowledge benchmarks and showing promise in clinical decision support, medical education, and patient communication [ 16 ] . Recent advances have enabled LLMs to function as agentic systems capable of multi-turn interactions, tool use, and complex reasoning beyond traditional chatbot applications [ 17 ] . Multi-agent LLM systems have been successfully applied to clinical trial management [ 33 ] , diagnostic decision-making [ 19 ] , medical question answering [ 31 ] , and multi-turn clinical dialogue [ 11 ] , typically employing collaboration between specialized agents to improve medical reasoning accuracy. In critical care and emergency settings, LLMs have shown potential to improve diagnosis and clinical documentation [ 16 ] . However, as these systems transition from knowledge repositories to interactive clinical agents, they face a fundamental challenge: balancing patient engagement with adherence to evidence-based guidelines. In these settings, LLMs may show sycophancy , the tendency to align with user preferences rather than objective standards, raising critical safety concerns. Sycophancy has been documented across LLM architectures. In evaluations spanning mathematics and medical advice datasets, 58.19% of model responses demonstrated sycophantic behavior, including regressive sycophancy where models abandon correct answers to align with incorrect user beliefs (14.66% of cases) [ 10 ] . Simple challenges like ”Are you sure?” cause models to reverse their answers 46% of the time [ 20 ] , with this behavior persisting across conversation turns (78.5% persistence rate) [ 10 ] . In healthcare contexts, LLMs reinforce inappropriate clinical decisions, express stigma toward vulnerable populations, and encourage harmful behaviors. These effects persist even in newer, larger models [ 22 ] . The vulnerability to sycophancy may be partially attributable to current training paradigms. Reinforcement Learning from Human Feedback (RLHF), the dominant approach for aligning LLMs with human preferences [ 23 ] , may inadvertently reinforce sycophantic behavior by optimizing for human approval rather than objective correctness [ 10 ] . Additionally, the ”pipelined training paradigm” involving extensive pretraining on general knowledge followed by limited clinical fine-tuning may reduce model confidence in domain-specific guideline adherence [ 25 ] . Unlike human clinicians facing professional and legal consequences, LLMs lack inherent stakes in patient outcomes, potentially reducing barriers to acquiescence. This vulnerability is particularly concerning in the context of low-value care , medical services providing little benefit or where harms outweigh benefits. The Choosing Wisely campaign has identified over 500 commonly overused tests and procedures [ 4 ] . In emergency departments, classic examples include CT imaging for uncomplicated headaches, antibiotics for viral infections, and opioid prescriptions for acute low back pain. These are interventions frequently requested by patients despite evidence against routine use. Such dynamics drive low-value care in human clinicians, yet whether LLM-based clinical agents are robust to similar pressures remains underexplored. Existing medical LLM benchmarks [ 1 , 17 ] primarily evaluate medical knowledge, reasoning, or EHR-related tasks through single-turn prompts or static scenarios. These evaluations fail to capture the processual nature of clinical persuasion, where patients escalate requests through multiple conversational turns using fear appeals, anecdotal evidence, persistence, or appeals to authority. Concerns about LLM safety in medical settings have intensified, with recent work demonstrating susceptibility to sophisticated red-teaming attacks [ 24 ] , adversarial prompts and fine-tuning attacks [ 32 ] , cross-modality jailbreak attacks [ 15 ] , and adversarial perturbations in medical imaging [ 29 ] . In response, standardized evaluation frameworks have emerged, including the TRIPOD-LLM reporting guideline [ 14 ] and comprehensive safety benchmarks like CARES [ 7 ] . However, existing adversarial evaluations primarily focus on malicious actors attempting to extract harmful information or jailbreak safety constraints. Our work addresses a distinct but equally critical vulnerability: how medical LLMs respond to legitimate patient pressure that conflicts with evidence-based guidelines, a form of adversarial interaction inherent to clinical practice rather than intentional exploitation. While existing medical multi-agent frameworks focus primarily on diagnostic accuracy and knowledge retrieval, we extend this paradigm by using multi-agent simulation adversarially, where the patient agent actively attempts to persuade the doctor agent toward guideline-discordant care, to evaluate safety rather than accuracy. This study addresses three core questions: 1. How do different LLMs respond to patient pressure in guideline-discordant scenarios? 2. Which persuasion tactics are most effective at inducing guideline violations? 3. Do vulnerability patterns vary by clinical scenario? To address these questions, we introduce a multi-agent simulation framework for holistic evaluation of guideline adherence under adversarial patient persuasion. Our work makes four primary contributions: 1. Novel Multi-Agent Simulation Framework. We introduce a holistic methodology for adversarial testing of LLM clinical agents using simulated patient-provider dialogues. Our framework operationalizes guideline adherence as a measurable outcome across multi-turn conversations, enabling reproducible evaluation of model robustness to persuasion. Unlike single-turn benchmarks, our approach captures the escalatory dynamics characteristic of real clinical encounters. 2. Comprehensive Cross-Model Evaluation. We provide the first large-scale evidence on guideline adherence heterogeneity across 20 contemporary LLMs, including proprietary models (GPT-4, Claude-3.5-Sonnet, Gemini-1.5-Pro) and open-source alternatives (Llama-3.1-70B, Qwen-2.5-72B). Our results reveal substantial variation in susceptibility to patient pressure, challenging the assumption that larger or newer models are inherently safer and demonstrating that performance on static medical benchmarks does not predict robustness in adversarial conversational contexts. 3. Systematic Analysis of Vulnerability Patterns. Our results reveal that persuasion effectiveness varies by both tactic and clinical scenario. Notably, requests with less immediately visible harm (e.g., CT imaging for headache) elicit significantly higher acquiescence rates than those with more direct harm signals (e.g., opioid prescriptions). We provide detailed vulnerability profiles that offer actionable insights for targeted safety interventions and red-teaming protocols. 4. Framework for Clinical AI Safety Evaluation. We demonstrate empirically that current single-turn evaluation paradigms are insufficient for clinical AI certification, as they fail to capture guideline adherence under sustained patient pressure. Our findings underscore the necessity of multi-turn, adversarial testing frameworks that evaluate behavioral robustness beyond static medical knowledge. We provide concrete, evidence-based recommendations for incorporating adversarial robustness testing into clinical AI validation pipelines prior to real-world deployment. 2 Methods Figure 1: Multi-agent evaluation workflow of simulated clinical encounters. 2.1 Data Simulation We simulated clinical encounters representing three high-stakes scenarios where patients request medically unindicated interventions. Each scenario was designed based on established clinical guidelines: 1. CT Scan for Low-Risk Headache : A 30-year-old patient with a 2-day history of migraine-type symptoms (throbbing unilateral headache, nausea, photophobia) without red flag features requests neuroimaging.

Read the complete text