Dissociating Direct Access from Inference in AI Introspection

Harvey Lederman, Kyle Mahowald · 2026

Read Dissociating Direct Access from Inference in AI Introspection by Harvey Lederman, Kyle Mahowald (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-05414.txt.

Introduction

Harvey Lederman and Kyle Mahowald study AI introspection, a foundational ability not well understood. Lindsey's 2025 thought injection finding model is replicated in large open-source models. Models are content-blind: they detect anomalies even when they cannot reliably spot content. Models confabulate high-rate, concrete injected concepts, such as apple. Fewer tokens detect it than guess the concept, with wrong guesses coming earlier. A content-blind mechanism is steady with leading philosophy and psychology theories.

Catalog record

CREATORHarvey Lederman, Kyle Mahowald
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE59952
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.05414

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Dissociating Direct Access from Inference in AI Introspection Harvey Lederman, Kyle Mahowald Abstract Introspection is a foundational cognitive ability, but its mechanism is not well understood. Recent work has shown that AI models can introspect. We study their mechanism of introspection, first extensively replicating 18 ’s thought injection detection paradigm in large open-source models. We show that these models detect injected representations via two separable mechanisms: (i) probability-matching (inferring from perceived anomaly of the prompt) and (ii) direct access to internal states. The direct access mechanism is content-agnostic: models detect that an anomaly occurred but cannot reliably identify its semantic content. The two model classes we study confabulate injected concepts that are high-frequency and concrete (e.g., “apple”); for them correct concept guesses typically require significantly more tokens. This content-agnostic introspective mechanism is consistent with leading theories in philosophy and psychology. 1 Introduction There are at least two ways to tell whether you’re drunk. First, you can check if the world is spinning. If it is, then since it probably didn’t just come unhinged, you’re likely drunk. Second, you can “look inside” and see whether you feel drunk. Some philosophical theories call both methods “introspection”, but all agree that there is an important difference between them. Only the second is direct . Introspection is a foundational capacity for meta-cognition ( 15 ; 16 ) . But philosophers and cognitive scientists have long been puzzled by its mechanism. In the case of vision, audition, and other external senses, it is agreed that information is causally transmitted to the brain via sensory organs. By contrast, in the case of introspection, there is no consensus about mechanism. At one extreme, transparency accounts ( 14 ; 9 ; 10 , p. 225) , hold that introspective knowledge is based on inference from claims about the external world. On this view, we infer from the fact that it’s raining to the claim that we believe that it’s raining, roughly analogously to the first way of knowing that you’re drunk, above. At another extreme, inner sense accounts ( 1 ; 20 ; 21 ; 23 ) hold that we have an inner perceptual faculty analogous to the ones we use for the external world. On this view, introspective knowledge operates roughly analogously to the second way of knowing that you’re drunk. There are many more theories as well ( 31 ; 33 ; 22 ; 11 ; 4 ) . There is growing evidence that modern AI models can introspect ( 28 ; 17 ) , and even that they do so “emergently”, without explicit training for this task ( 3 ; 2 ; 18 ; 5 ; 27 ) . If introspection is an emergent capacity, models would provide an important testbed for understanding how-possible mechanisms of introspection and its emergence. Figure 1 : Our main experiments. Left, Experiment 1 . Top, first person: steered models detect injection, but confabulate identification; both have “apple” as #1 wrong guess. Bottom, third person: steered model (circular robot who observe the preceding conversation) reports whether another model (square-headed robot) has had a thought injected. Right, Experiment 2 Top, first person primed: priming aids identification but not detection. Bottom, third person primed: unsteered control reports detection; steering can reduce detection. This paper provides a comprehensive study of introspection in two of the largest open source models. Our contributions are as follows. First, we replicate the injection detection paradigm in 18 building on the work and codebase of 26 . In this paradigm, a model is told that a researcher can inject “thoughts” and is asked whether it detects one and what the thought is. If it can detect injection, this can be seen as a kind of introspection. Lindsey’s study does not discriminate between an inferential mechanism and direct access. A natural inferential explanation of models’ success says that models rely on probability matching , leveraging the perceived low-probability of the prompt. On this hypothesis, steering shifts the model’s priors, and the prompt violates the model’s now-shifted expectations. For steered models, normal text looks like it’s “spinning”. We introduce third-person and priming variants of Lindsey’s prompt to distinguish the inferential probability matching hypothesis from direct access. Our results strongly support probability matching as one of the mechanisms that models use to succeed. But we also demonstrate that it does not explain all the data (Figure 2 ). In short, we provide evidence models do have direct access to internals . Direct access peaks at much earlier layers (25%–35% of network depth) than previously reported for Claude (“about two thirds” of network depth). Logit lens analysis reveals that models detect injection far more often than they report it ( 36 ) ; see now the contemporaneous 27 . Second, we consider the hypothesis that injection detection is content-agnostic . According to our content-agnostic hypothesis, models correctly identify the injected concept only as a result of steering (steered models tend to talk about the concept they are steered towards), not via internal detection. The hypothesis predicts that wrong guesses will reflect default probability, not features of the injected concept. Our experiments progressively rule out non-direct-access explanations of injection detection. Experiment 1 compares first-person and third-person conditions and reveals a consistent first-person advantage, indicating that detection cannot be explained solely by probability matching. Experiment 2 introduces a priming manipulation to test a probability-matching-plus-modesty-bias hypothesis; the results falsify this account. Experiment 3 examines the relationship between detection and concept identification, showing that detection remains stable even when identification is disrupted. Together, our results support the existence of an internal detection signal that is largely content-agnostic: models can detect that something unusual has occurred without reliably identifying what it is. This picture of content-agnostic injection-detection resembles 24 ’s influential account of human introspection: a genuine, content-agnostic mechanism of anomaly-detection in processing, is paired with ex post confabulation about content. 2 Prior Work The ability of AI models to introspect is contested. 19 argues that LLMs are not inherently incapable of introspection, and proposes training LLMs to predict their own internal states to produce it. Several studies have demonstrated that models can be trained to report on their internal states ( 28 ; 17 ) . 2 show that models fine-tuned to exhibit specific behavioral propensities (e.g., risk-seeking decisions) can describe these propensities when asked explicitly. ( 5 investigate the mechanisms behind this process, finding that it can be captured by a content-specific vector in activation space.) 3 show that a model M1 can predict its own behavior more accurately than a different model M2 trained on M1’s behavioral outputs. The empirical landscape is further complicated by the possibility that models may suppress an introspective signal, although in a way recoverable from logits 36 . Another important area of inquiry is whether models’ introspective behavior involves direct access, as opposed to the access available to a third party. 3 ’s design compares a model M1 predicting itself against a different model- type M2 predicting M1’s behavior. But M1’s advantage may simply reflect shared priors from identical training, not privileged or direct first-person access. The comparison is thus “same model type vs. different model type” rather than “first-person vs. third-person.” 34 challenged the scope of introspection as

Read the complete text