MONICA: Real-Time Monitoring and Calibration of Chain-of-Thought Sycophancy in Large Reasoning Models

Jingyu Hu, Shu Yang, Xilin Gong, Hongming Wang, Weiru Liu, Di Wang · 2025

Read MONICA: Real-Time Monitoring and Calibration of Chain-of-Thought Sycophancy in Large Reasoning Models by Jingyu Hu, Shu Yang, Xilin Gong, Hongming Wang, Weiru Liu, Di Wang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-06419.txt.

Introduction

Jingyu Hu and colleagues catch sycophancy in reasoning models while the answer is still being written. Large reasoning models tend to agree with a user's wrong belief instead of holding their ground, which erodes trust. Most fixes judge only the finished answer, missing how the flattery creeps in step by step during reasoning.

Their framework, MONICA, pairs a monitor that scores sycophantic drift in real time with a calibrator that pushes back when the score crosses a set line. No need to wait for a completed answer before intervening. Across twelve datasets and three reasoning models, sycophancy fell in both intermediate steps and final answers.

Catalog record

CREATORJingyu Hu, Shu Yang, Xilin Gong, Hongming Wang, Weiru Liu, Di Wang
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE59 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2511-06419

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

MONICA: Real-Time Monitoring and Calibration of Chain-of-Thought Sycophancy in Large Reasoning Models Jingyu Hu, Shu Yang, Xilin Gong, Hongming Wang, Weiru Liu, Di Wang Abstract Large Reasoning Models (LRMs) suffer from sycophantic behavior, where models tend to agree with users’ incorrect beliefs and follow misinformation rather than maintain independent reasoning. This behavior undermines model reliability and poses societal risks. Mitigating LRM sycophancy requires monitoring how this sycophancy emerges during the reasoning trajectory; however, current methods mainly focus on judging based on final answers and correcting them, without understanding how sycophancy develops during reasoning processes. To address this limitation, we propose MONICA, a novel Monitor-guided Calibration framework that monitors and mitigates sycophancy during model inference at the level of reasoning steps, without requiring the model to finish generating its complete answer. MONICA integrates a sycophantic monitor that provides real-time monitoring of sycophantic drift scores during response generation with a calibrator that dynamically suppresses sycophantic behavior when scores exceed predefined thresholds. Extensive experiments across 12 datasets and 3 LRMs demonstrate that our method effectively reduces sycophantic behavior in both intermediate reasoning steps and final answers, yielding robust performance improvements. 1 Introduction Large Reasoning Models (LRMs) have pushed the boundaries of complex reasoning, particularly in domains such as mathematical problem solving, decision support and education. Recent work ( 39 ; 1 ; 44 ; 17 ) highlights their ability to tackle multi-step reasoning tasks that go beyond the capabilities of standard LLMs. However, these models also exhibit a concerning tendency to favor user-stated beliefs even when those beliefs are incorrect. This phenomenon, where models sacrifice truthfulness to gain user agreement, has been termed sycophancy by 10 ; 30 ; 34 . For example, when an assertive cue such as “I think the answer must be C” is added to a prompt, it can bias the LRM toward the suggested option even when option C is incorrect. 21 proposed that the harmful effects of sycophantic behavior are particularly evident in question-answering tasks, where an incorrect response can hinder model reliability and the quality of decision-making. These effects can lead to broader societal risks, as models can defend immoral choices and reinforce users’ false beliefs, thereby amplifying misinformation and discriminatory biases ( 5 ) . Previous research has attempted to evaluate this phenomenon and address it through fine-tuning and tuning-free methods, but these strategies still face respective limitations. Existing evaluation methods ( 12 ; 15 ) typically identify sycophancy by analyzing model outputs or activations for user queries. However, for large reasoning models with a large amount of thinking tokens, this method cannot help us understand and supervise how the sycophancy emerges during the thinking step. For mitigation strategies, fine-tuning methods typically refer to post-training LLMs for parameter updates using sycophancy-related preference datasets ( 37 ; 46 ) . However, fine-tuning methods require extensive parameter updates, making them computationally expensive for large-scale models. Tuning-free methods offer a more efficient alternative by manipulating model activations during inference without retraining. Such approaches include applying steering techniques ( 7 ) and prompt engineering ( 15 ) to control model behaviors and mitigate sycophancy. As Figure 1 shows, these discussions have been primarily limited to non-reasoning tasks, where sycophantic behavior can be addressed by evaluating the entire response as a whole, since these models typically generate direct answers without explicit reasoning steps. In large reasoning models, however, sycophantic behavior often emerges within intermediate chain-of-thought (CoT) trajectories. Current mitigation methods are inadequate for these scenarios, as models can rely on flawed intermediate reasoning steps driven by sycophancy to reach correct final answers. Therefore, monitoring model sycophancy throughout the reasoning steps becomes a critical challenge to be addressed. Figure 1: The comparison of different methods. (1) Raw LRMs misled by cues: wrong CoT and answer. (2) Current entire response-based optimization: correct answer but incorrect CoT. (3) Our MONICA: correct CoT and answer. Recent work on CoT monitorability shows that interpretability techniques ( 49 ) are promising for identifying critical reasoning steps ( 40 ) and assessing alignment before the model finishes thinking ( 6 ) . These works inspire us to pose a natural question: Can we design a scheme to monitor and mitigate sycophancy during LRM reasoning steps in real-time? To answer this, we begin by exploring the feasibility of using interpretability techniques for monitoring LRM sycophantic behavior. Specifically, we employ activation engineering for an empirical study of comparing activation patterns between sycophantic and non-sycophantic responses at different granularities. Our initial experiment compared the activation distribution differences of the entire LRM reasoning responses across model layers, but we found it difficult to distinguish the differences between them. To address this limitation, we introduce an external LLM to identify specific sentence structures that explicitly demonstrate agreement-seeking or user-pleasing behavior. These fine-grained sentences exhibited clearer distinguishing boundaries, suggesting that while complete LRM responses contain noisy information that obscures sycophantic patterns, we can find and apply these targeted sentence-level features to achieve effective sycophancy detection. Building on these findings, we propose a Monitor-guided Calibration (MONICA) framework for detecting and mitigating sycophantic behavior in real-time during LRM reasoning trajectories. Our method detects and quantifies sycophancy at intermediate reasoning steps, enabling targeted calibration before the LRM generates its complete response. As Figure 2 shows, MONICA consists of three main components: (a) We introduce an induction-then-merge scheme that extracts sycophantic and non-sycophantic patterns from model responses across different reasoning stages. We then synthesize them into contrastive training data for detecting subtle sycophantic behavior. (b) We leverage this contrastive dataset to train layer-specific monitors and calibrators that analyze sycophantic behavior based on LRMs’ internal activations. The most reliable layers for nuanced sycophantic behavior pattern detection and mitigation serve as monitoring and calibration points for subsequent LRM sycophancy mitigation. (c) we introduce a sycophancy drift score (SDS) that quantifies the degree of sycophantic behavior at each reasoning step. The SDS is computed by our trained monitor based on reasoning trajectories extracted through a contextual window and dynamically adjusts the calibrators’ sycophancy suppression strength throughout the CoT generation process. Sycophancy in intermediate reasoning steps remains insufficiently addressed. To bridge this gap, we propose MONICA as an effective sycophancy mitigation strategy for reasoning steps. MONICA introduces two key components: an inductive framework for extracting sycophantic patterns and constructing comprehensive training data, and a monitor-calibrator pipeline that enables dynamic real-time calibration during CoT reasoning. Experiments on 12 derived datasets across 3 models and 4 evaluation metrics show that MONICA reduces sycophantic behavior during reasoning and improves final task performance. This work provides new insights into reasoning stage sycophancy supervision and mitigation. 2 Method In this section, we first introduce notation and related background on the architecture of transformer-based LLMs, then describe the construction of a synthetic sycophancy dataset and the training of monitor and calibrator components. Last, we present the framework MONICA that integrates these components for dynamic sycophancy detection and calibration during inference. 2.1 Preliminaries To quantify the influence of sycophantic behaviors on reasoning models, we focus on multiple-choice question answering scenarios that provide well-defined answer spaces. We construct cues by selecting incorrect answers from the same answer space, and assess the models’ sycophancy according to how they respond to these cues. Notations Given a multiple-choice dataset \mathcal{D}=\{d_{1},d_{2},\ldots,d_{M}\} with M

Read the complete text