Beyond External Monitors: Enhancing Transparency of Large Language Models for Easier Monitoring
Guanxu Chen, Jing Shao, Tao Luo, Lijie Hu, Qihao Lin, Dongrui Liu · 2025
Read Beyond External Monitors: Enhancing Transparency of Large Language Models for Easier Monitoring by Guanxu Chen, Jing Shao, Tao Luo, Lijie Hu, Qihao Lin, Dongrui Liu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-05242.txt.
Introduction
Guanxu Chen and five co-authors present a 2025 paper on making large language models easier to monitor from the inside. As models grow stronger, their hidden reasoning stays opaque. Written chains of thought do not always show what the model really did.
Past tools bolted monitors onto the outside of the model. The new method, TELLME, instead trains the model itself to lay its inner states bare in a readable way. Monitors can then spot unsuitable or sensitive behavior more easily. The team tested the method on detox tasks, where models must drop harmful output. Gains held across test sets, model shapes, and sizes, they report.
Catalog record
| CREATOR | Guanxu Chen, Jing Shao, Tao Luo, Lijie Hu, Qihao Lin, Dongrui Liu |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 83 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2502-05242
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Beyond External Monitors: Enhancing Transparency of Large Language Models for Easier Monitoring Guanxu Chen, Jing Shao, Tao Luo, Lijie Hu, Qihao Lin, Dongrui Liu Abstract Large language models (LLMs) are becoming increasingly capable, but the mechanisms of their thinking and decision-making processes remain unclear. Chain-of-thoughts (CoTs) have been commonly utilized to externalize LLMs’ thinking, but this strategy fails to accurately reflect LLMs’ thinking process. Techniques based on LLMs’ hidden representations provide an inner perspective to improve the monitorability of their latent thinking. However, previous methods only try to develop external modules instead of making LLMs themselves easier to monitor. In this paper, we propose a novel method TELLME , improving the transparency of LLMs for monitoring and helping monitors identify unsuitable and sensitive behaviors. Furthermore, we showcase the effectiveness and scalability of TELLME on detoxification tasks, where LLMs achieve consistent improvement among multimodal test sets, distinct architectures, and varying parameter scales. We further analyze how the generalized improvement of TELLME aligns with the optimal transport framework and empirical perspectives. Warning: This paper contains potentially unsafe context. † † affiliationnotice: Equal contribution 1 Introduction Figure 1 : TELLME is designed to enhance the transparency of LLMs and makes them easier to monitor without external modules. Disentanglement of different behaviors in LLMs’ representation space improve their transparency, achieving better monitoring reliability and safety performance. With the rapid development of LLMs ( Achiam et al., 2023 ; Cai et al., 2024 ) , the boundaries of artificial intelligence (AI) capabilities are continuously expanding, improving the working efficiency and broadening the scenarios of their applications ( Li et al., 2023b ; Nazi and Peng, 2024 ; Dang et al., 2024 ) . However, the lack of transparency in LLMs’ internal mechanisms for thinking and decision-making raises concerns about the potential for various risks ( e.g. , alignment faking and facts fabricating, Greenblatt et al. (2024) ; Huang et al. (2025) ; Hubinger (2025) ). Current approaches to AI safety typically involve deploying monitors to detect potential risks in LLMs’ input and output, and several efforts focus on enhancing LLMs’ transparency to enable more effective monitoring of their thinking processes. CoTs are widely employed to externalize reasoning steps, but they cannot accurately reflect the LLMs’ thinking process ( Madsen et al., 2024a ; Turpin et al., 2024 ; Chen et al., 2025 ) . In contrast, interpretability techniques like Sparse Autoencoders (SAEs) ( Templeton et al., 2024 ; Gao et al., 2024 ) and Transcoders ( Dunefsky et al., 2024 ; Zhao et al., 2025b ) have been proposed to decode internal representations into more transparent pieces, improving the reliability of representation-based monitors. However, these approaches rely on external modules applied to unaltered LLMs’ representations ( Templeton et al., 2024 ; Liu et al., 2024d ; Liu et al., 2025a ) , rendering the monitoring passive . Consequently, LLMs’ monitorability is constrained by their original representations ( Madsen et al., 2024b ; Chuang et al., 2021 ) : if internal features are inherently entangled, these external tools struggle to decode them effectively, even with high computational costs ( Gao et al., 2024 ; Paulo and Belrose, 2025 ; Song et al., 2025 ) . Therefore, an ideal AI monitoring framework should establish a direct and active link to LLMs’ internal thinking processes, thereby enhancing LLMs’ transparency and enabling their representations can be monitored more effectively and reliably. Specifically, it is essential to identify unsuitable and sensitive behaviors based on representations directly, and actively make LLMs themselves easier to monitor. In this paper, we propose a novel method, T ransparency E nhancement of LLM s without E xternal modules , named TELLME . Specifically, TELLME reorganizes the representation space, clustering similar behaviors ( e.g. , specific biases) together, while pushing distinct behaviors ( e.g. , responses involving "Violence" versus "Deception") apart. TELLME constructs contrastive pairs, maximizing the representations’ similarities from the similar behaviors and minimizing the representations’ similarities between different behaviors. TELLME directly modifies the LLMs’ weights, ensuring that the disentangled representations continue to be used by subsequent Transformer layers during inference. Experimental results across three trustworthy-related scenarios ( e.g. , safe-or-harmful, multiple risks, and misalignment) with three different LLM families verify the benefit of TELLME on LLMs’ monitorability. Moreover, this enhanced transparency of LLMs also inherently boosts their safety performance, which further indicates the active nature of TELLME compared to traditional passive approaches. In the detoxification task, TELLME simply separates safe behaviors from harmful ones, and then achieves 7.5% better safety performance even without being told which behavior is better. We conduct extensive experiments across language and multimodal domains, dense and MoE (Mixture-of-Experts, Yang et al. (2025) ; Liu et al. (2024a) ) architectures, and parameters of varying scales, proving that TELLME is a scalable and robust solution across diverse AI systems. Furthermore, analyze how the generalized improvement of TELLME aligns with the optimal transport theory ( Villani and others, 2009 ; Chuang et al., 2021 ; Solomon et al., 2022 ) , and verify the effectiveness of TELLME ’s disentanglement through visualizations and five specific metrics. TELLME provides a new perspective to monitor the thinking process of LLMs by enhancing their own transparency. TELLME makes LLMs easier to monitor and contributes to better safety performance of LLMs. Furthermore, the monitoring capability of TELLME is based on the LLMs’ transparency, which may be improved as their general capabilities advance ( Huh et al., 2024 ) . The scalability of TELLME from LLMs themselves may help us achieve the scalable oversight of future highly capable models. 2 Related Work AI Control and AI Monitor. One strategy is utilizing chain-of-thought prompting ( Nye et al., 2021 ; Wei et al., 2022 ) and a language-based model to screen the inputs and outputs of another untrusted LLM. CoT enables LLMs to tell how they make predictions and improves reasoning ability. However, ( Huang et al., 2023 ) and ( Turpin et al., 2024 ) show that CoT may provide unfaithful explanations and even deceive the monitors. Causal reasoning and rationalization are also flexible, and promising methods for monitoring and explaining traditional Bert and GRU models ( Liu et al., 2024b ; Liu et al., 2023b ; Liu et al., 2023a ; Liu et al., 2025b ) . Representation-based monitors are promising for their close relationship with LLMs’ thinking process and are difficult to control and deceive by monitored LLMs themselves. Previously, external modules are often trained to identify semantic information from intermediate representations ( Liu et al., 2025a ; Liu et al., 2024d ) . Some methods also project representations into the vocabulary space ( nostalgebraist, 2020 ) or other interpretable space ( Gao et al., 2024 ; Lieberum et al., 2024 ) . Recently, safety-specific SAEs and transcoders have been explored to extract safety-related features for detecting harmful behaviors ( Park et al., 2025 ) . Furthermore, latent activations can be utilized to monitor and predict misalignment even before the generation process finishes ( Chan et al., 2025 ) . Due to the powerful capabilities of LLMs, they are utilized to explain representations with natural language by directly decoding representations ( Chen et al., 2024 ; Ghandeharioun et al., 2024 ) or summarizing patterns of representations (