Multi-Turn Neural Transparency: Surfacing Neural Activations Improves User Calibration to LLM Behavioral Drift

Sheer Karny, Anthony Baez, Pat Pataranutaporn · 2026

Read Multi-Turn Neural Transparency: Surfacing Neural Activations Improves User Calibration to LLM Behavioral Drift by Sheer Karny, Anthony Baez, Pat Pataranutaporn (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-15455.txt.

Introduction

Sheer Karny, Anthony Baez, and Pat Pataranutaporn ask whether showing people a language model's inner workings helps them judge shifting behavior in long chats. Their display turns activation patterns into scores for six traits, shown as a sunburst chart, a drift panel, and attention cues. In a trial with 246 people, the moving display improved prediction and judgment, curbed overconfidence, and beat a still version without losing ease of use.

Catalog record

CREATORSheer Karny, Anthony Baez, Pat Pataranutaporn
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE66658
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.15455

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Multi-Turn Neural Transparency: Surfacing Neural Activations Improves User Calibration to LLM Behavioral Drift Sheer Karny, Anthony Baez, Pat Pataranutaporn Abstract . Chatbot behavior is often opaque to users, as responses can shift unpredictably across a conversation, drifting toward sycophancy, toxicity, or other unsafe responses. This can leave users vulnerable, either being misled by overly agreeable AI or manipulated by a harmful chatbot that no longer behaves as intended. To address this, we introduce multi-turn neural transparency , an interface that surfaces an LLM’s internal neural activations in real time to help users anticipate and recognize how behaviors change across turns. We construct behavioral vectors for six personality traits using methods from mechanistic interpretability, identifying directions in activation space that correlate with trait expression ( R 2 ≥ 0.9 R^{2}\geq 0.9 ) via contrastive system prompts, and visualize trait expression using a sunburst and drift panel that updates at each turn. In a randomized controlled study (N = 246), participants predicted trait expression from a system prompt alone, then rated observed behavior after interacting with the chatbot for both assistant and role-play personas. We find that participants without visualization struggled to accurately evaluate traits (RMSE ≈ \approx 0.6-0.7), while the inclusion of neural transparency significantly improved both anticipation and evaluation compared to no visualization (d = -0.34 to -0.49). The multi-turn dynamic visualization additionally outperformed the static single-turn visualization on holistic evaluation of model behavior (d = -0.32). Transparency also reduced overconfidence: participants without visualization grew more confident despite no gain in accuracy. These findings suggest that surfacing internal model representations to everyday users is a meaningful step toward more transparent and informed human-AI interaction. Figure 1. (Left) Sunburst visualization of behavioral trait activations for behavioral state. (Middle Top) Overall layout of Neural Transparency interface with sunburst, drift panel, and chat. (Middle Bottom) Drift panel displaying changes in selected trait expression over entire conversation. (Right) Example conversation with chatbot demonstrating how the sunburst changes at each conversation turn. 1. Introduction LLM-based chatbots are widely deployed in diverse settings which span personal use, productivity at work, and even emotional support. Despite such widespread adoption, the underlying mechanisms of these systems remain unknown for everyday users. Furthermore, recent work which surfaces insights from such mechanisms is often limited to static, single-turn interaction scenarios ( 38 ) . However, static scenarios fail to reflect typical engagement with LLMs over extended, multi-turn conversations where model behavior can shift unpredictably ( 21 ) . Under such conditions, models may drift towards unsafe behaviors such as sycophancy or toxicity in ways that typical analyses cannot capture. In extreme circumstances, long-form interaction can even weaken safety guardrails and contribute to delusional spirals and other psychologically unsafe behaviors ( 27 ; 37 ; 3 ) . For users to be both generally informed and able to anticipate harmful behavioral drift, they require a calibrated mental model of AI behavior. These user mental models of AI must shift with the drift, yet current interfaces impede this by providing no direct insight into this dynamic. However, recent advances in mechanistic interpretability (MI) can allow such direct insight into the model behaviors that shape human-AI interaction. One such finding from MI that may be used to surface AI behaviors is how LLMs represent features—such as personality—as linear directions in their activation space ( 6 ) . These representations are causally linked to behavior and can predict behavioral drift in dynamic multi-turn conversations ( 24 ) . Few have attempted to leverage such mechanistic insights for human-AI interfaces ( 6 ; 18 ) , especially in a multi-turn setting where models are subject to significant behavioral shifts. We situate our work in this critical intersection of dynamic, causal interpretations of LLM behaviors and intuitive human-AI interfaces. Toward this goal, we offer multi-turn neural transparency as a design principle which bridges intuitive MI interfaces with dynamic interaction settings. To evaluate multi-turn neural transparency, we conducted a randomized controlled study (N=246) in which participants predicted chatbot behavior from a system prompt, then evaluated the behaviors they observed. Users interacted with both a typical assistant persona and a more variable role-playing persona. Using this paradigm, we investigate the impact of multi-turn transparency on human-AI interaction with the following research questions: (1) RQ1: How does neural transparency—in both static (single-turn) and dynamic (multi-turn) forms—affect users’ ability to anticipate and evaluate LLM behavior compared to no visualization? (2) RQ2: Does dynamic, multi-turn neural transparency improve user calibration beyond a static single-turn snapshot, and does this benefit depend on the behavioral variability of the persona? (3) RQ3: How does neural transparency affect users’ self-reported confidence in their ability to predict model behavior? In addressing these questions, we make four major contributions to the field of interface design for human-AI interaction: (1) A Mechanistically-Informed Multi-Turn Interface. We design and implement a dynamic interface that translates LLM activations into an accessible sunburst visualization and drift panel, enabling non-technical users to monitor behavioral state and trajectory across conversation turns. (2) Behavioral Vectors and Validated Scoring. We introduce a method for constructing and rigorously validating linear representations of LLM behavioral traits via contrastive system prompts for a multi-turn setting, achieving R^{2}\geq 0.90 across all six traits. (3) Empirical Evidence for Neural Transparency. A controlled user study (N=246) demonstrating that multi-turn neural transparency significantly improves anticipation and evaluation of AI behavior and reduces overconfidence in self-reported ability to predict AI behavior. (4) Neural Transparency as a Design Paradigm. We articulate neural transparency as a vision for the future of human-AI interaction: one where the internal states of AI systems are not hidden but visible, trackable, and actionable by the people who use them, bridging mechanistic interpretability and everyday human-AI interaction. 2. Related Work This work sits at the intersection of human-AI interaction and mechanistic interpretability. We survey three areas that directly inform our approach: transparency and explainability methods for surfacing LLM behavior to users, mechanistic interpretability research characterizing how behaviors are encoded in LLMs, and previously existing interfaces that have translated these internal representations into accessible visualizations. These respective bodies of work motivate the need for a multi-turn transparency interface grounded in the model’s internal mechanisms. 2.1. Transparency in AI Explainable AI (XAI) studies how to create faithful and intuitive explanations for model behavior ( 42 ; 13 ; 14 ; 12 ; 11 ) , including in language-based models such as LLMs ( 34 ) . One aspect of explainability is transparency, where AI explanations, whether mechanistic or post-hoc, are shown to users to better calibrate trust and improve safety ( 22 ) . Empirical user studies have examined the effects of showing an LLM’s explanations alongside its outputs ( 36 ) , finding that explanations improve user trust, though the effect is highly dependent on how they are presented. Relatedly, making chain-of-thought reasoning visible, a technique used by modern LLMs to reason through problems step-by-step, was also shown to increase users’ trust, decision confidence, and willingness to adopt AI advice when the reasoning is correct, while unfaithful or inconsistent rationales can reduce trust and miscalibrate reliance ( 39 ) . However, even unfaithful explanations ( 11 ) produced similar effects, underscoring the importance of faithfulness in AI transparency. Hippo ( 31 )

Read the complete text