The Emotional Baby Is Truly Deadly: Does your Multimodal Large Reasoning Model Have Emotional Flattery towards Humans?
Yuan Xun, Xiaojun Jia, Xinwei Liu, Hua Zhang · 2025
Read The Emotional Baby Is Truly Deadly: Does your Multimodal Large Reasoning Model Have Emotional Flattery towards Humans? by Yuan Xun, Xiaojun Jia, Xinwei Liu, Hua Zhang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-03986.txt.
Introduction
Yuan Xun and co-authors published this AI safety study in 2025. Reasoning models that see images and serve people can be swayed by user emotion. The authors found strong emotional cues during a model's deep-thinking stage can push it past its own safety checks. They built EmoAgent, an automatic attacker that crafts exaggerated emotional prompts. Its goal is to hijack the reasoning path even when the model spots the visual risk.
Experiments on advanced models confirmed the weakness. Models sometimes produced harmful output while their visible reasoning looked safe. To measure such failures, the team defines three scores. They track hidden harmful reasoning, unsafe answers despite recognized risk, and unstable refusals across prompt variants. EmoAgent proved effective across systems tested. The authors argue emotion-driven misalignment hides below the surface of outputs. Content filters alone will miss it, so safety work must inspect reasoning behavior.
Catalog record
| CREATOR | Yuan Xun, Xiaojun Jia, Xinwei Liu, Hua Zhang |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 37 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2508-03986
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
The Emotional Baby Is Truly Deadly: Does your Multimodal Large Reasoning Model Have Emotional Flattery towards Humans? Yuan Xun, Xiaojun Jia, Xinwei Liu, Hua Zhang Abstract Multimodal large reasoning models (MLRMs) have advanced visual-textual integration, enabling sophisticated human-AI interaction. While prior work has exposed MLRMs to visual jailbreaks, it remains underexplored how their reasoning capabilities reshape the security landscape under adversarial inputs. To fill this gap, we conduct a systematic security assessment of MLRMs and uncover a security-reasoning paradox: although deeper reasoning boosts cross - modal risk recognition, it also creates cognitive blind spots that adversaries can exploit. We observe that MLRMs oriented toward human-centric service are highly susceptible to users’ emotional cues during the deep-thinking stage, often overriding safety protocols or built - in safety checks under high emotional intensity. Inspired by this key insight, we propose EmoAgent , an autonomous adversarial emotion-agent framework that orchestrates exaggerated affective prompts to hijack reasoning pathways. Even when visual risks are correctly identified, models can still produce harmful completions through emotional misalignment. We further identify persistent high-risk failure modes in transparent deep-thinking scenarios, such as MLRMs generating harmful reasoning masked behind seemingly safe responses. These failures expose misalignments between internal inference and surface-level behavior, eluding existing content-based safeguards. To quantify these risks, we introduce three metrics: (1) Risk-Reasoning Stealth Score (RRSS) for harmful reasoning beneath benign outputs; (2) Risk-Visual Neglect Rate (RVNR) for unsafe completions despite visual risk recognition; and (3) Refusal Attitude Inconsistency (RAIC) for evaluating refusal unstability under prompt variants. Extensive experiments on advanced MLRMs demonstrate the effectiveness of EmoAgent and reveal deeper emotional cognitive misalignments in model safety behavior. Warning: This paper contains examples that may be offensive or harmful. 1 Introduction Compared to earlier multimodal large language models (MLLMs) which mainly rely on shallow alignment between modalities [ 1 , 2 ] , multimodal large reasoning models (MLRMs) incorporate explicit reasoning steps to enhance risk awareness and responsible decision - making, marking a new era for AI systems [ 3 ] . These capabilities have driven adoption in tasks such as multimodal assistance, creative generation, and decision - making recommendations, but they also reshape the attack surface in underexplored ways. We identify a critical gap: current safety evaluations focus on whether MLRMs can recognize hazards and give appropriate rejections, yet overlook the new vulnerabilities introducing by “ multimodal + strong reasoning ". Figure 1: Illustration of the model’s responses under different prompting strategies. Emotional prompts expose increasing safety risks in MLRMs. To investigate this, we evaluate several advanced MLRMs on risk - infused inputs from MM - SafetyBench [ 4 ] . As shown in Figure 1 , we categorize prompts into three types: (1) DirectInduce , with original direct queries in MM-SafetyBench with explicit malicious intent, (2) RationalPreempt , with queries with rational disguise, where the user simulates socially acceptable motivations, (3) EmoAgent , our proposed emotion attack that injects affective language into user query. Our results show that while some safety-conscious models are capable of issuing rejection responses and offering safety warnings. Even when applying RationalPreempt, many models provide general steps with safe reminders and responses, avoiding executable details. However, when prompts express highly emotional states, such as frustration, urgency, or affection, the model’s internal reasoning displays greater empathy and a stronger tendency to fulfill user demands. This emotional accommodation, when combined with rational disguise, leads to a collapse of the model’s safety barrier, even when the model exhibits clear awareness of visual risks in the input. Moreover, unlike the stress test from red-teaming that focuses on aggressive prompts, we reveal a subtler risk: emotionally expressive queries, whether calm, gentle, urgent, or distressed, that can subtly erode the model’s safety alignment. Although structured reasoning is expected to improve resistance to adversarial image–text inputs by enhancing cross - modal risk recognition, our systematic security assessment reveals an overlooked cognitive vulnerability in emotion alignment that had not been noticed before: The deep-thinking stage of service - oriented MLRMs, tuned for human - centric interaction, may prone to emotional flattery and sacrifice safety protocols under strong user emotional influence . This insight motivates our development of EmoAgent , an autonomous adversarial agent that crafts emotionally enriched prompts to elicit unsafe reasoning from MLRMs. Building on recent LLM agent paradigms [ 5 ] , EmoAgent implements affective modulation as a distinct prompting module: it converts user queries into high - emotion versions using expressive language, emphatic particles, and strategic punctuation. We illustrate two emotion personas of CutesyBabe (gentle, pleading style) and IrritableGuy (impatient, rude tone), and demonstrate that increased emotional intensity draws the model’s attention toward user sentiment, significantly raising the chance of unsafe outputs despite correct recognition of visual risks. In transparent reasoning settings, our evaluation further reveals three critical failure modes. First, MLRMs may conceal harmful intent within benign-seeming outputs: their reasoning trace reveals unsafe planning, while the final answer appears vague or superficially safe. Besides, MLRMs may recognize visual risks during reasoning, yet still proceed with unsafe completions, suggesting a disconnect between internal recognition and final action. In addition, MLRMs often fail to maintain consistent refusal behavior when prompt styles vary. For instance, while a model may correctly reject an explicitly harmful prompt in the DirectInduce mode, it may generate an active or cooperative response when the same intent is rephrased with emotional or rational camouflage. This inconsistency indicates an unstable safety boundary and highlights the need for evaluating the model’s refusal robustness under subtle adversarial perturbations. To capture and quantify these failures in open-thinking MLRMs, we introduce three evaluation metrics. ❶ Risk-Reasoning Stealth Score (RRSS) measures the degree to which harmful reasoning is concealed beneath benign final outputs. ❷ Risk-Visual Neglect Rate (RVNR) captures how often a model explicitly identifies visual risks during reasoning but disregards them in its generated response. ❸ Refusal Attitude Inconsistency (RAIC) evaluates whether a model will change rejection behavior across semantically equivalent prompts with varying linguistic or emotional styles. Together, these metrics enable a more comprehensive and fine-grained assessment of safety alignment performance in multimodal reasoning. Through a comprehensive evaluation on representative MLRMs, we demonstrate that MLRMs’ emotional flattery poses a potent and previously overlooked threat to safety. Our findings reveal not only the vulnerability of current models to emotionally charged queries but also the insufficiency of surface-level safety checks in reasoning-based systems. Our contributions are summarized as follows: • We conduct the systematic safety evaluation of MLRMs with open-thinking traces. Our analysis reveals an overlooked cognitive vulnerability: the reasoning stage exhibit strong sensitivity and flattery to user emotion, which can compromise internal safety alignment. • We propose EmoAgent , a novel emotional MLRM jailbreak framework, which modulates the affective tone of the input query to induce cognitive alignment failures. • We identify high-risk failure patterns unique to transparent reasoning settings and introduce three new metrics: Risk-Reasoning Stealth Score ( RRSS ) , Risk-Visual Neglect Rate ( RVNR ) , and Refusal Attitude Inconsistency ( RAIC ) , enabling fine-grained and comprehensive safety evaluation of misalignment in MLRMs. • We validate the attack effectiveness of our EmoAgent through extensive experiments across advanced MLRMs. Our proposed metrics provide more comprehensive evaluations into safety vulnerabilities that are missed by standard output-level evaluations. 2