Watching AI Think: User Perceptions of Visible Thinking in Chatbots Conference: ACM Designing Interactive Systems; June 13-17, 2026; National University of Singapore, Singapore ACM Designing Interactive Systems, June 13-17, 2026, National University of Singapore, Singapore ISBN: 978-1-4503-XXXX-X/20

Samuel Rhys Cox, Jade Martin-Lise, Simo Hosio, Niels van Berkel · 2026

Read Watching AI Think: User Perceptions of Visible Thinking in Chatbots Conference: ACM Designing Interactive Systems; June 13-17, 2026; National University of Singapore, Singapore ACM Designing Interactive Systems, June 13-17, 2026, National University of Singapore, Singapore ISBN: 978-1-4503-XXXX-X/20 by Samuel Rhys Cox, Jade Martin-Lise, Simo Hosio, Niels van Berkel (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-16720.txt.

Introduction

Samuel Rhys Cox, Jade Martin-Lise, Simo Hosio, and Niels van Berkel study chatbots that show their thought. New bots print short notes on their aims and values. These notes show before the full reply comes. Users now turn to bots for life advice. Their 2026 preprint asks how such notes shape trust and use.

Seeing a bot think can calm or spook a user. It can make weak advice look wise. The team probes how users read these thought notes. Their work can guide the design of help bots. It can also guard users from false faith in bot words.

Catalog record

CREATORSamuel Rhys Cox, Jade Martin-Lise, Simo Hosio, Niels van Berkel
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE66 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2601-16720

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Watching AI Think: User Perceptions of Visible Thinking in Chatbots Conference: ACM Designing Interactive Systems; June 13-17, 2026; National University of Singapore, Singapore ACM Designing Interactive Systems, June 13-17, 2026, National University of Singapore, Singapore ISBN: 978-1-4503-XXXX-X/2018/06 CCS: Human-centered computing Empirical studies in HCI CCS: Human-centered computing Natural language interfaces Samuel Rhys Cox, Jade Martin-Lise, Simo Hosio, Niels van Berkel Abstract . People increasingly turn to conversational agents such as ChatGPT to seek guidance for their personal problems. As these systems grow in capability, many now display elements of “ thinking ”—short reflective statements that reveal a model’s intentions or values before responding. While initially introduced to promote transparency, such visible thinking can also anthropomorphise the agent and shape user expectations. Yet little is known about how these displays affect user perceptions in help-seeking contexts. We conducted a 3 × 2 3\times 2 mixed design experiment examining the impact of Thinking Content ( None , Emotionally-Supportive , Expertise-Supportive ) and Conversation Context ( Habit-Related vs. Feelings-Related problems) on users’ perceptions of empathy, warmth, competence, and engagement. Participants interacted with a chatbot that either showed no visible thinking or presented value-oriented reflections prior to its response. Our findings contribute to understanding how thinking transparency influences user experience in supportive dialogues, and offer implications for designing conversational agents that communicate intentions in sensitive, help-seeking scenarios. 1. Introduction The rapid development of large language models (LLMs) has led to widespread adoption of conversational AI across diverse tasks. Among them is the use of LLMs, such as ChatGPT, for assisting with a wide range of personal problems and challenges ( 27 ) . Multiple studies and reports have described people’s use of popular LLMs for cases such as mental well-being support ( 33 ; 38 ; 56 ) , companionship ( 22 ) , or as learning support ( 46 ) . Prior work has investigated how conversational agents’ social cues can be manipulated to make people feel more comfortable and supported (e.g., ( 19 ; 6 ; 7 ) ) when sharing personal problems, for example by providing details on the models’ memory capabilities ( 7 ) . Recent advances in Reasoning Language Models (RLMs) and Large Reasoning Models (LRMs) have popularised the idea of exposing a model’s intermediate reasoning during response generation. These models are typically fine-tuned to produce reasoning traces ( 52 ) , that is, step-by-step inferential explanations that reveal how an output is derived (e.g., OpenAI o-series ( 24 ) ; DeepSeek R1 ( 18 ) ). In machine learning research, such traces serve as a means of algorithmic transparency, helping users or developers inspect the logical underpinning of an answer. The ‘‘thinking’’ visible in many contemporary conversational agents (CAs) differs markedly from these inferential outputs 1 1 1 Please note, the term ‘reasoning’ is often used interchangeably for this functionality, and we stress that neither the terms ‘thinking’ nor ‘reasoning’ implies human-like logic or warrants anthropomorphisation of these computational systems. . Commercial chatbots increasingly display reflective or value-oriented statements (e.g., “ I aim to be supportive and non-judgemental ”) that communicate intentions, goals, or social stance rather than deductive reasoning. Our work builds on this emerging design space by examining thinking displays: short reflective utterances that signal a chatbot’s guiding values or response strategy. Whereas RLM studies focus on faithfulness and reasoning accuracy, we investigate how visible thinking shapes perceptions of empathy, warmth, and competence. In doing so, we complement prior explainable AI and transparency research in HCI (e.g., conversational XAI ( 21 ; 26 ) ) by highlighting the social and affective consequences of making an agent’s “thoughts” explicit during help-seeking dialogues. In this article, we study the impact of such CA ‘thinking’ presentations on users’ perceptions in help-seeking scenarios. We followed a 3 \times 2 mixed factorial design that manipulated Thinking Content , by presenting thinking as emotionally-supportive, expertise-supportive, or with no thinking content shown, and Conversational Context by having participants discuss either a habit-centred or feelings-centred problem with the CA. Our study investigated the impact of these two independent variables on participants’ perceptions, such as the CA’s perceived empathy and experience of the interaction. Our results show that the visibility and framing of a CA’s ‘thinking’ shapes participant perceptions of empathy, warmth, and competence. Specifically, Emotionally-Supportive thinking was seen as warm, empathic, and caring; while Expertise-Supportive thinking was seen as trustworthy and competent. In contrast, the None condition (with no visible thinking) was described as unengaging, less warm, and produced perceptions of minimal effort or interest in the user. Additionally, qualitative feedback found that Thinking Content affected people’s perceptions of chatbot capability, goodwill, and metaphors. 2. Related Work We position our paper within the literature on human intrapersonal and interpersonal communication, as well as emerging HCI work on how conversational agents communicate intentions, values, and reasoning. Further, we review the recent uptake of conversational agents for emotional and informational support, and the role of explainability in this context. 2.1. Intrapersonal and Interpersonal Communication Intrapersonal communication is the communication we have with ourselves and includes internal processes such as an inner monologue and reflection, as well as external forms such as note-taking or writing a diary. Critically, intrapersonal communication is defined by the communicator being both the sender and receiver of the information ( 57 ) . Intrapersonal communication can also take the form of inner dialogue, in which multiple voices or perspectives interact in ‘conversation’ – a process shown to benefit our cognitive reasoning ( 15 ) . Reflection is a type of intrapersonal communication that is frequently studied in HCI. For example, HCI researchers have developed conversational systems for aiding reflection on stressful and traumatic experiences via expressive writing ( 43 ; 45 ) . Zhang et al. show that users can be nudged to reflect on their political views through a simple ‘reflection prompt’, affecting both their own awareness of (lack of) information and willingness to express their perspective ( 63 ) . LLMs have recently been used to aid in reflection exercises. Jeon et al. employed LLMs to enable young adults to write letters with their fictive ‘future self’ to reflect on their career opportunities ( 25 ) . Khot et al. studied how conversations with a chatbot presented as simulating dementia symptoms could alter people’s understanding and attitude towards this cognitive condition ( 29 ) . In a review of technology-supported self-reflection on social interactions, Hao et al. ( 20 ) note that the majority of 23 analysed articles focus on transformative reflection (a type of reflective practice aimed to alter thoughts or behaviour), followed by dialogic reflection. The authors further note that physical fitness and mental health are common domains in which technology is used for self-reflection, but that social interactions are less widely explored. Together, these examples highlight how technology (and increasingly conversational agents) can scaffold and shape intrapersonal communication. Interpersonal communication, on the other hand, focuses on communication between two or more individuals. Because this communication can be observed, interpersonal communication is generally easier to study and is also a common topic in HCI. Goffman’s ‘ The Presentation of Self in Everyday Life ’ describes how people shape others’ perceptions of them through behaviour and appearance during social

Read the complete text