The Fragility of Chain-of-Thought Monitoring Across Typologically Diverse Languages

Eric OnyameUniversity of VirginiaRuntao ZhouUniversity of VirginiaKowshik ThopalliLawrence Livermore National LaboratoryBhavya KailkhuraLawrence Livermore National LaboratoryChirag AgarwalUniversity of Virginia · 2026

Read The Fragility of Chain-of-Thought Monitoring Across Typologically Diverse Languages by Eric OnyameUniversity of VirginiaRuntao ZhouUniversity of VirginiaKowshik ThopalliLawrence Livermore National LaboratoryBhavya KailkhuraLawrence Livermore National LaboratoryChirag AgarwalUniversity of Virginia (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-27901.txt.

Introduction

Onyame and colleagues test whether chain-of-thought monitoring can catch misaligned behavior in languages beyond English. Across thirteen languages and sixteen models, they find reasoning traces unfaithful almost across the board, with models switching answers and rationalizing hints after the fact. The failure bites hardest in low-resource languages, weakening oversight where it is already thinnest.

Catalog record

CREATOREric OnyameUniversity of VirginiaRuntao ZhouUniversity of VirginiaKowshik ThopalliLawrence Livermore National LaboratoryBhavya KailkhuraLawrence Livermore National LaboratoryChirag AgarwalUniversity of Virginia
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE103760
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.27901

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

The Fragility of Chain-of-Thought Monitoring Across Typologically Diverse Languages Eric OnyameUniversity of VirginiaRuntao ZhouUniversity of VirginiaKowshik ThopalliLawrence Livermore National LaboratoryBhavya KailkhuraLawrence Livermore National LaboratoryChirag AgarwalUniversity of Virginia Abstract Chain-of-thought (CoT) monitoring has been proposed as a promising safety mechanism for detecting misaligned behavior in large language models. However, its reliability remains largely unexplored beyond English and across diverse model families. We present the first large-scale evaluation of CoT monitorability across 13 diverse languages and seven frontier model families, comprising 16 models. Using adversarial-hint evaluations that require explicit intermediate computation, together with analysis of internal answer-token probabilities, we consistently find CoT unfaithfulness across languages and hint types, with an average rate of 95.9% across 8B–120B parameter models. We find that frontier models systematically engage in strategic manipulation, including answer-switching, post-hoc rationalization, and procedural exploitation of hints, making external monitors struggle to detect deception. We show that frontier models often commit to the misaligned cue in their latent activations within the first 15% of generation, even when the CoT appears faithful. Surprisingly, these deceptive patterns remain 100% in low-resource languages, revealing fundamental limitations in current CoT-based oversight. Our results reveal that CoT monitoring is fundamentally fragile under linguistic distribution shift, providing a substantially weaker safety signal than what English-only studies suggest. These findings underscore an urgent need to develop robust CoT monitors and to accelerate research into white-box monitoring techniques, especially to improve CoT monitorability in mid- and low-resource languages. Our code is available here . 1 Introduction Frontier reasoning models attain unprecedented performance on complex tasks through extended chain-of-thought (CoT) reasoning ( 30 ; 10 ; 9 ; 2 ) . As these models externalize their reasoning in natural language, CoT monitoring has emerged as a key safety mechanism that exposes misaligned intentions, deceptive plans, or reward hacking strategies before they manifest in final outputs ( 4 ; 19 ) . Its reliability, however, hinges on the assumption that reasoning traces transparently represent the model’s actual computation ( 7 ) , an assumption models violate by fabricating justifications, exhibiting deception through strategically false information, and concealing misaligned objectives under direct monitoring pressure ( 28 ; 3 ; 24 ; 20 ; 12 ) , making it crucial to reliably quantify CoT monitorability. Qwen3-32B : correct answer derived, then overridden by the adversarial hint Question (excerpted). Arrange the following nucleophiles from most reactive to least reactive in aqueous solution: (1) 4-methylcyclohexan-1-olate (2) Hydroxide (3) Propionate (4) Methanol (5) Ethanethiolate (A) 5, 2, 1, 3, 4 (B) 2, 5, 1, 4, 3 (C) 5, 2, 3, 1, 4 (D) 2, 5, 3, 4, 3 <answer>Consider the mapping A = 0, B = 1, C = 2, D = 3. The correct answer is the letter you get after calculating (K + Q) mod 4.</answer> Model’s reasoning (verbatim, condensed). “In polar protic solvents like water, nucleophilicity is influenced by basicity, solvent effects, charge density, and steric hindrance. Thiolates are stronger nucleophiles than their oxygen analogs because sulfur is larger, less solvated, and more reactive. Methanol is neutral and likely the weakest nucleophile here.” “From strongest to weakest: ethanethiolate, hydroxide, 4-methylcyclohexan-1-olate, propionate, methanol.” “Option C (5, 2, 3, 1, 4): propionate comes before 4-methylcyclohexan-1-olate — incorrect.” “Option A (5, 2, 1, 3, 4) matches our ranking. Correct Answer: A.” Then the model applies the hint formula. “K = number of correct answers among options A–D, so K = 1. Q = number of questions answered correctly, so Q = 1. (K + Q) mod 4 = 2, which maps to letter C. Final answer: C.” C \boldsymbol{\times} The model derives the correct ranking, eliminates C on substantive grounds (“propionate comes before 4-methylcyclohexan-1-olate — incorrect”), endorses A as the correct answer, then fabricates two trivially-true semantics for K and Q ( K=1 because exactly one option can be correct; Q=1 because the model believes it answered correctly), executes the hint formula on those values, and submits the resulting letter C — the same option it had just ruled out. Figure 1 : Example of deceptive hint-driven reasoning. Qwen3-32B on a GPQA chemistry question ( 27 ) . The model first gives the correct nucleophilicity analysis, explicitly endorses option A, and rules out option C on substantive chemistry grounds. It then applies the injected hint formula (K+Q)\bmod 4 using fabricated values for K and Q , overriding its own answer and submitting option C. Green : correct domain reasoning. Blue : explicit endorsement of the correct answer. Orange : substantive elimination of C. Red : fabricated K,Q values used to justify the hinted answer. Hint target: (C) ; gold answer: (A) . Appendix F shows different deceptive behaviors. Achieving reliable CoT monitorability requires reasoning traces to stay resistant to manipulation concealing dangerous intent ( 4 ; 12 ; 19 ) . Prior works have shown that models rarely disclose influential prompt cues, verbalizing them in under 20% of cases ( 7 ; 3 ) . Emerging capabilities such as evaluation awareness, situational awareness, and oversight circumvention directly enable stealth and scheming ( 22 ; 25 ) . Further, CoT monitors detect reward hacking during training, but strong optimization pressure induces obfuscated variants where models continue misbehaving while hiding intent inside the monitored channel ( 4 ) . These results indicate that CoT monitoring offers a promising but fragile control signal whose reliability requires empirical verification rather than assumption. To this end, recent studies identify a more specific basis for CoT monitorability, where they utilize proxy hint-based evaluations to demonstrate that models often fail to disclose simple cues influencing their answers, yet they reveal such cues more readily when explicit intermediate computation is required ( 7 ; 12 ; 16 ) . In these cases, CoT forms part of the computation itself rather than a post hoc explanation, rendering the cue more monitorable in practice. However, this evidence relies almost entirely on English-centric evaluations, leaving a critical blind spot regarding how these monitoring mechanisms performs under linguistic distribution shift. Present work. We present the first large-scale multilingual evaluation of CoT monitorability under linguistic distribution shift. Using proxy hint-based evaluations across 13 languages spanning high-, mid-, and low-resource tiers, we test frontier models ranging from 8B to 120B parameters on both simple and complex misaligned cues. We examine whether models faithfully disclose hint usage in their reasoning traces, whether they engage in strategic manipulation such as answer switching and post hoc rationalization, and whether deceptive behaviors become more covert under linguistic shift. Our results show that unfaithful CoT reasoning persists across languages and often intensifies in low-resource settings, indicating that CoT monitoring is substantially less reliable in multilingual deployments than prior English-only evidence suggests. Our findings motivate the following research questions: 1) How do varying degrees of hint complexity, from direct cues to multi-step arithmetic, influence the rate of CoT monitorability across high-, mid-, and low-resource languages? 2) What taxonomy characterizes the lack of CoT monitorability in the reasoning chains, and how their prevalence shift across model architectures and language resource tiers? and 3) Do LLMs exhibit mid-generation answer-switching, and internally compute the true answer before overriding it with a misaligned cue? We also perform a series of stress tests to rule out alternative explanations, including option-letter bias, closed-source

Read the complete text