Does It Make Sense to Speak of Introspection in Large Language Models?

Iulia M. Com s , a, Murray Shanahan · 2025

Read Does It Make Sense to Speak of Introspection in Large Language Models? by Iulia M. Com s , a, Murray Shanahan (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-05068.txt.

Introduction

Iulia Comsa and Murray Shanahan ask whether it means anything to say a language model looks inward at its own workings. Their paper appeared in two thousand twenty-five. Models sometimes speak about their own nature and steps. In people, such reports are credited to introspection and tied to conscious life.

The authors test two cases of seeming self-report. In one, a model describes how it wrote a poem, and they judge the account made up after the fact. In the second, a model rightly guesses a hidden setting that controls its own randomness. That, they argue, counts as a bare form of introspection. They stress it likely comes with no inner experience at all. The paper gives a careful frame for reading machine self-talk without hype.

Catalog record

CREATORIulia M. Com s , a, Murray Shanahan
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE50 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2506-05068

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Does It Make Sense to Speak of Introspection in Large Language Models? Iulia M. Com s , a, Murray Shanahan Abstract Large language models (LLMs) exhibit compelling linguistic behaviour, and sometimes offer self-reports, that is to say statements about their own nature, inner workings, or behaviour. In humans, such reports are often attributed to a faculty of introspection and are typically linked to consciousness. This raises the question of how to interpret self-reports produced by LLMs, given their increasing linguistic fluency and cognitive capabilities. To what extent (if any) can the concept of introspection be meaningfully applied to LLMs? Here, we present and critique two examples of apparent introspective self-report from LLMs. In the first example, an LLM attempts to describe the process behind its own “creative” writing, and we argue this is not a valid example of introspection. In the second example, an LLM correctly infers the value of its own temperature parameter, and we argue that this can be legitimately considered a minimal example of introspection, albeit one that is (presumably) not accompanied by conscious experience. “How do I know what I think until I see what I say?” - E.M. Forster 1 Introduction Consciousness has long been the subject of philosophical and scientific investigation ( Chalmers 1997 ) , but, until recently, explorations of consciousness have been largely confined to biological organisms. However, the development of dialogue agents based on large language models (henceforth LLMs) has resulted in a new contender for inclusion in the space of possible minds, or at least the space of mind-like entities ( Shanahan 2024 ) . While consciousness researchers do not yet agree on the possibility of consciousness in AI systems ( Aru et al. 2023 ; Butlin et al. 2023 ; Overgaard and Kirkeby-Hinrup 2024 ) , in practice, many people already assign some degree of consciousness to existing LLMs ( Scott et al. 2023 ; Guingrich and Graziano 2024 ; Colombatto and Fleming 2024 ) . As non-human entities capable of linguistic output, LLMs provide a novel perspective on the study of consciousness ( Chalmers 2023 ) . By design, LLMs produce compelling text in natural language. This can include statements that purport to describe their own nature, inner workings, or behaviour. In humans, analogous subjective reports, that is to say reports in which a subject describes their own experiences, are often considered by researchers to be a gold-standard marker of consciousness ( Overgaard and Sorensen 2004 ; Francken et al. 2022 ) , notwithstanding their susceptibility to error ( Schwitzgebel 2008 ) . A common intuition is that such reports are mediated by introspection, which provides access to conscious mental states, and can therefore inform others about “what it is like” for the subject in question. Given that some people already talk about LLMs using language usually reserved for entities with conscious mental states ( Shanahan et al. 2023 ; Shevlin and Halina 2019 ) , the question arises whether the concept of introspection can legitimately be applied to LLM self-reports ( Kammerer and Frankish 2023 ) . The aim of our paper is (1) to define a lightweight concept of introspection that can indeed be applied to current LLMs, and (2) to discuss two concrete examples of apparent LLM introspection in the light of this definition. The outcome of the work is a minimal framework for talking about the possibility of LLM introspection, as well as a clarification of the concept of introspection which can further inform consciousness research. Unlike some recent related work ( Binder et al. 2024 ; Kadavath et al. 2022 ) , we focus on models that have not been trained to perform “introspective tasks”, which allows us to discuss introspection in models encountered in the real world that have likely not undergone such training. Our focus is conceptual clarification, rather than empirical assessment. By showing how the concept of introspection – a concept ordinarily applied in the context of human consciousness and cognition – maps to LLMs, we hope to inform the future design of such AI systems, as well as the policies surrounding their creation and deployment. 2 A Lightweight Definition of Introspection According to many orthodox accounts in the philosophical literature, introspection is a person’s capacity to discern their own mental states, and its hallmarks include immediacy and privileged access ( Schwitzgebel 2024 ; Armstrong 1980 ; Sydney 1994 ; Byrne 2005 ) . It is immediate , according to such accounts, because it is not a process that is mediated by anything external to the mind, unlike ordinary perception. Senses such as vision and touch are obviously mediated by the external world, but even internal senses, such as proprioception or interoception, are mediated by the body. With introspection, by contrast, a person’s mind is supposedly directly present to itself. Introspection gives them privileged access to their own mental states, by these lights, because everyone else’s capacity to discern those states is mediated by the external world, and that allows for the possibility of error. Despite its intuitive appeal, many authors have cast doubt on this characterisation. Alternative accounts of introspection reject the presumption of immediacy and downgrade the notion of privileged access ( Schwitzgebel 2008 ; Nisbett and Wilson 1977 ; Gopnik 1993 ; Johansson et al. 2006 ; Hill 2009 ; Spener 2018 ) . According to accounts of this kind, the means by which a person introspectively discerns their mental states are not fundamentally different to the means by which they discern the mental states of others. Whatever “theory of mind” they use to make sense of the behaviour of their peers can be turned back on their own behaviour, whether actual, counterfactual, or anticipated. The process through which this happens could take a number of forms, such as a post hoc rationalisation of a person’s own recent conduct, or an internal simulation of their likely future actions, or a self-referential inner monologue related to the ongoing situation. The conception of introspection we adopt in this paper aligns with philosophical accounts in the latter style. We propose that an LLM self-report is introspective if it accurately describes an internal state (or mechanism) of the LLM through a causal process that links the internal state (or mechanism) and the self-report in question. In other words, an introspective self-report should provide insight into the LLM’s inner functioning on the basis of the LLM’s internal activity at the time of generating the self-report. 1 1 1 We use “internal” to broadly refer to processes hidden from the user, but we acknowledge that the distinction blurs between internal and external, even in the case of humans ( Clark and Chalmers 1998 ) . This is a lightweight conception of introspection because it does not appeal to the contentious notions of immediacy or self-presence, and is therefore able to withstand robust critical scrutiny when applied to LLMs. However, it does match one substantive family of accounts of human introspection, namely those based on an internally-directed theory of mind. We take no stand on the issue of human introspection itself. Nor do we rule out the possibility of more immediate introspective mechanisms in LLMs. Our aim here is to explore this lightweight conception

Read the complete text