Auditorily Embodied Conversational Agents: Effects of Spatialization and Situated Audio Cues on Presence and Social Perception Conference: Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems; April 13–17, 2026; Barcelona, Spain Proceedings of the 2026 CHI Conference on Human
Yi Fei Cheng, Jarod Bloch, Alexander Wang, Andrea Bianchi, Anusha Withana, Anhong Guo, Laurie M. Heller, David Lindlbauer · 2026
Read Auditorily Embodied Conversational Agents: Effects of Spatialization and Situated Audio Cues on Presence and Social Perception Conference: Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems; April 13–17, 2026; Barcelona, Spain Proceedings of the 2026 CHI Conference on Human by Yi Fei Cheng, Jarod Bloch, Alexander Wang, Andrea Bianchi, Anusha Withana, Anhong Guo, Laurie M. Heller, David Lindlbauer (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-22082.txt.
Introduction
Yi Fei Cheng and co-authors give conversational agents a voice that seems to come from somewhere. Embodiment, having a perceptible body, raises how present an agent feels. That presence is usually built visually, through avatars. But visual channels are not always available, as with headphones or glasses without displays.
The team explores auditory embodiment instead: spatialized sound and situated audio cues that place the agent in the listener's space. Their 2026 preprint tests how those cues affect the sense of presence. Voice-only interfaces are everywhere, from earbuds to car assistants, so knowing how sound alone can ground an agent has direct design value.
Catalog record
| CREATOR | Yi Fei Cheng, Jarod Bloch, Alexander Wang, Andrea Bianchi, Anusha Withana, Anhong Guo, Laurie M. Heller, David Lindlbauer |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 77 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2601-22082
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Auditorily Embodied Conversational Agents: Effects of Spatialization and Situated Audio Cues on Presence and Social Perception Conference: Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems; April 13–17, 2026; Barcelona, Spain Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems (CHI ’26), April 13–17, 2026, Barcelona, Spain DOI: 10.1145/3772318.3791794 ISBN: 979-8-4007-2278-3/2026/04 CCS: Human-centered computing Empirical studies in HCI CCS: Human-centered computing Sound-based input / output Yi Fei Cheng, Jarod Bloch, Alexander Wang, Andrea Bianchi, Anusha Withana, Anhong Guo, Laurie M. Heller, David Lindlbauer Abstract . Embodiment can enhance conversational agents, such as increasing their perceived presence. This is typically achieved through visual representations of a virtual body; however, visual modalities are not always available, such as when users interact with agents using headphones or display-less glasses. In this work, we explore auditory embodiment . By introducing auditory cues of bodily presence – through spatially localized voice and situated Foley audio from environmental interactions – we investigate how audio alone can convey embodiment and influence perceptions of a conversational agent. We conducted a 2 ( spatialization : monaural vs. spatialized ) × 2 ( Foley : none vs. Foley ) within-subjects study, where participants (n=24) engaged in conversations with agents. Our results show that spatialization and Foley increase co-presence, but reduce users’ perceptions of the agent’s attention and other social attributes. Figure 1. We explore how auditory embodiment influences perceptions of conversational agents. While agents are often embodied visually, such modalities may not always be available, for instance, when interacting through headphones (left) . We investigate whether embodiment can be conveyed solely through audio, using spatialized voice and situated Foley sounds. For example, an agent may be represented as seated next to the user while typing on a laptop (middle) , or as picking up toy blocks across the room (right). The visual depiction of the agent is for illustration purposes only. A three-panel illustration showing a participant interacting with an auditorily embodied conversational agent in their living room. In the left panel, the participant sits on a couch, reading a magazine. In the middle panel, the participant continues reading while casually conversing with the agent, whose voice is spatialized to sound as if it is sitting next to them; Foley audio of typing reinforces its presence. In the right panel, the agent’s voice and Foley cues shift across the room as it interacts with toy blocks on a table, conveying its movement and activities even though it is represented only through sound. † † cc-license: by 1. Introduction In recent years, conversational agents (e.g., Google’s Gemini Live, OpenAI’s ChatGPT voice mode) have advanced rapidly in their ability to engage in naturalistic dialogue. They increasingly demonstrate human-like behaviors ( 16 ) and can respond to spoken inputs in real time ( 22 ) . These capabilities have driven the growing popularity of applications and experiences that support conversational interactions, ranging from interactive museum guides ( 60 ) to services for social engagement and emotional support ( 25 ) . In the design of conversational agents, an important factor shaping user experiences is the way the agent is represented ( 79 ) . A common approach to representation is embodiment , where agents are given a bodily form ( 11 ) . Prior work has shown that embodying an agent can yield a range of advantages, from providing richer multimodal communication cues to support user tasks ( 40 ) to enhancing the agent’s perceived social presence ( 2 ) . More importantly, embodiment shapes how people behave around agents, for better or worse. For instance, embodiment can foster greater trust in the agent ( 49 ) . This can be beneficial in domains such as education where trust may support student engagement ( 81 ) ; however, this can also lead to unrealistic expectations of the agent’s abilities ( 21 ) . These implications have motivated extensive research on how best to represent conversational agents. In both prior research and commercial systems, embodiment has typically been realized through visual representations ( 77 ) , such as presenting avatars in Virtual or Augmented Reality (VR/AR). Yet, visual output modalities are not always available or desirable. A user may interact with an agent through earphones while their device is in their pocket, such as when walking or cooking. Recent smart glasses, such as the Ray-Ban Meta Glasses ( 65 ) , are also display-less and rely primarily on auditory interactions. In these scenarios, current methods and understandings of agent embodiment may not be applicable. In this work, we explore auditory embodiment of conversational agents. Specifically, we ask: can embodiment be introduced through audio alone, and how does this influence user perceptions of the agent? In real-world social contexts, people are often co-present with others they cannot see, such as when someone is behind them. Nevertheless, their presence can still be sensed through (1) the spatial location of their voice and (2) incidental sounds generated by bodily movements and interactions with the environment (e.g., footsteps or rustling clothing). Similarly, in film, auditory realism is often enhanced through the use of everyday Foley sounds that accompany on-screen actions. Drawing on how humans naturally use incidental bodily and environmental sounds to infer others’ presence in shared space, as well as the use of everyday Foley sounds to convey situated actions in film, we investigate whether such auditory cues can serve as mechanisms for supporting the embodiment of conversational agents. In particular, we examine how spatialization and situated Foley audio that represent an agent’s movement and actions shape user perceptions and behaviors. As an initial exploration of auditory embodiment , we study its effects in the context of casual social interactions with conversational agents. Prior research has long examined embodied agents in social settings ( 11 ) , with more recent work focusing on conversational agents as companions ( 25 ) . Our work aims to provide insights into the design of conversational agents for social use, particularly in how they may be represented auditorily. To this end, we conducted a within-subjects controlled study (n=24) in which participants engaged in and evaluated conversations with a conversational agent. In each conversation, the agent’s voice was rendered either spatialized or monaural (non-spatial). For each spatialization condition, we further varied whether the agent’s audio was presented alone or accompanied by Foley sounds. Our results show that spatializing the agent’s audio and adding Foley contribute to stronger feelings of co-presence. However, the addition of Foley also reduces attention and message comprehension, and leads to a negative social impression. Overall, we contribute empirical results showing how spatialization and Foley influence the social presence and perception of conversational agents. Through our results, we discuss opportunities and challenges of auditorily embodying conversational agents, highlighting considerations for the design of future systems. 2. Related Work Our study builds on prior research on auditory interfaces, embodied conversational agents, and social presence. 2.1. Auditory Interfaces Over the last 40 years, there has been persistent interest in auditory interfaces and interactions ( 29 ) . Early work on sound-based interactions, such as earcons ( 7 ) and auditory icons ( 29 ) , primarily focused on information delivery. Subsequent research has significantly broadened