Grounded Misunderstandings in Asymmetric Dialogue: A Perspectivist Annotation Scheme for MapTask

2025

Read Grounded Misunderstandings in Asymmetric Dialogue: A Perspectivist Annotation Scheme for MapTask (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-03718.txt.

Introduction

Nan Li, Albert Gatt, and Massimo Poesio build an annotation scheme that tracks what each speaker in a dialogue thinks a phrase refers to. In the MapTask corpus, partners guide each other by landmarks on maps that do not quite match. Partners can believe they agree while picturing different places, a grounded misunderstanding.

Separate labels for speaker and listener readings let analysts trace how understanding grows, splits, and gets repaired. A constrained language model pipeline produced thirteen thousand labeled references with reliability scores. Full misunderstandings turn out rare once wording variants are unified. Count mismatches between partners, though, reliably seed divergence, showing how apparent agreement can hide a split reference.

Catalog record

YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE38 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2511-03718

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Grounded Misunderstandings in Asymmetric Dialogue: A Perspectivist Annotation Scheme for MapTask Abstract Collaborative dialogue relies on participants incrementally establishing common ground, yet in asymmetric settings they may believe they agree while referring to different entities. We introduce a perspectivist annotation scheme for the HCRC MapTask corpus (Anderson et al., 1991) that separately captures speaker and addressee grounded interpretations for each reference expression, enabling us to trace how understanding emerges, diverges, and repairs over time. Using a scheme-constrained LLM annotation pipeline, we obtain 13k annotated reference expressions with reliability estimates and analyze the resulting understanding states. The results show that full misunderstandings are rare once lexical variants are unified, but multiplicity discrepancies systematically induce divergences, revealing how apparent grounding can mask referential misalignment. Our framework provides both a resource and an analytic lens for studying grounded misunderstanding and for evaluating (V)LLMs’ capacity to model perspective-dependent grounding in collaborative dialogue. Keywords: reference expression, common ground, misunderstanding, collaborative task, annotation scheme Grounded Misunderstandings in Asymmetric Dialogue: A Perspectivist Annotation Scheme for MapTask Nan Li, Albert Gatt, Massimo Poesio Utrecht University, Utrecht, The Netherlands {n.li, a.gatt, m.poesio}@uu.nl Abstract content 1. Introduction In everyday collaborative dialogue, speakers and addressees continuously coordinate their understanding of what is being discussed Clark and Brennan 1991 ; Pickering and Garrod 2004 . A central mechanism for this coordination involves reference expressions (REs)—linguistic forms used to pick out entities in the shared context. Utterances become part of the common ground only after being recognized and acknowledged by the addressee Clark and Wilkes-Gibbs 1986 ; Clark and Schaefer 1989 ; Clark and Brennan 1991 , a process known as grounding . Traditional approaches to reference resolution assume that once grounding is achieved via explicit or tacit confirmation, both interlocutors successfully refer to the same entity. However, this assumption can fail in asymmetric settings where participants have access to different information. Such asymmetric settings were the focus of the HCRC MapTask ( Anderson et al. 1991 ; see Figure 1 for an example), where two participants navigate a route using slightly different maps. While participants in such settings show systematic adaptation behaviors to address information asymmetry Bard et al. 2000 ; Viethen et al. 2011a ; Viethen et al. 2011b ; Healey et al. 2018 , making full misunderstandings rare, few studies have examined the personal interpretations of REs by the two interlocutors and the subtle divergences that may persist even after apparent grounding. These limitations also exist in the MapTask annotations (discussed in Section 4.1 ), making them difficult to be used to identify misunderstandings. To understand how interlocutors achieve and sometimes fail to achieve mutual understanding, we must first capture each participant’s personal interpretation. Taking both perspectives allows us to quantify where apparent alignment hides subtle misalignments and to measure how frequent such misunderstandings actually are under different landmark discrepancy settings. Figure 1: A simplified example of misunderstandings from MapTask dialogues. Left : the instruction giver ’s map contains two instances of parked van and the intended target is circled. Middle : the instruction follower ’s map shows only one parked van at the bottom that is shared with the giver . Right : the giver says “go to the parked van ,” the follower acknowledges but grounds another instance, leading to correction (“No!”). Red highlights the reference expression. Prior frameworks for modeling dialogue states offer rich tools to trace how shared understanding is established Poesio and Traum 1997 ; Cooper et al. 1999 ; Matheson et al. 2000 ; Lascarides and Asher 2009 ; Schlangen and Skantze 2011 ; Ginzburg 2012 . However, these symbolic, frame-based representations have not been widely integrated into reference-level annotation schemes, partly because they are difficult for large language models (LLMs) to reproduce in a constrained, schema-driven manner. We introduce an annotation framework based on the MapTask annotations that explicitly captures the divergence between the speaker’s intended and the addressee’s interpreted referents during collaborative dialogue. Our scheme uses a hierarchy of attributes to represent personal interpretations and incremental grounding, making it both linguistically principled and operational for LLM annotation. Applying this framework to MapTask dialogues with GPT-5 OpenAI 2025 , we scale annotation across the entire MapTask corpus. While full misunderstandings remain uncommon, our analysis reveals systematic patterns of understanding state transitions. These findings not only shed light on how diverging interpretations emerge and resolve in collaborative tasks but also establish a benchmark for evaluating language-only and vision-language models’ grounding abilities. Our research questions are: RQ1 Is it possible to develop a personal-interpretation annotation scheme capture nuanced understanding states for REs in collaborative tasks like MapTask? RQ2 Given conflicting personal interpretations, can we trace how understanding evolves across turns until the successful grounding? RQ3 Can LLMs, under a schema-constrained protocol, reliably annotate personal interpretations; and how does their output inform future evaluation of (V)LLMs on grounded dialogue? Our contributions include: • We operationalize reference expression grounding as a graded, perspective-dependent alignment process, and propose a annotation scheme at the RE level. • We present a scheme-constrained LLM-in-the-loop annotation pipeline of MapTask dialogues, producing 13k annotated REs with reliability estimates. • We release analysis protocols that associate RE-level annotations with dual-perspective interpretations and incremental understanding states, enabling quantitative study of misalignment dynamics in MapTask dialogues and informing future evaluation of LLMs and VLMs on common grounding. The remainder of this paper is structured as follows. Section 2 reviews related work. Section 3 introduces the MapTask corpus and its challenges for reference grounding. Section 4 details our annotation scheme and LLM-in-the-loop annotation pipeline. Section 5 analyzes understanding state distributions and transitions. Finally, Section 6 concludes with implications and future directions. 2. Related work Collaborative Reference and Grounding Reference expressions are central to establishing and maintaining common ground in interactive dialogue Clark and Wilkes-Gibbs 1986 ; Clark and Schaefer 1989 . Successful mutual understanding is constrained by many factors, e.g., communication medium, interactive alignment at different linguistic levels Clark and Brennan 1991 ; Pickering and Garrod 2004 . Previous work reveals that speakers do not always fully accommodate addressee knowledge, exhibiting egocentric biases even when explicitly aware of information asymmetry Horton and Keysar 1996 ; Keysar et al. 2000 ; Lane et al. 2006 This theoretical foundation has motivated NLP tasks to study grounded reference in collaborative settings with asymmetric information. The HCRC MapTask Anderson et al. 1991 and iMap corpus Louwerse et al. 2007 both instantiate navigation tasks where participants have slightly different maps, eliciting rich adaptation behavior Bard et al. 2000 ; Viethen et al. 2011a ; Viethen et al. 2011b . More recent work includes the OneCommon Corpus Udagawa and Aizawa 2019 , a partially-observable environment for both participants. These approaches typically treat grounding as a single end state rather than tracking divergent personal interpretations throughout the dialogue. Current VLMs have excelled at some object identification tasks Yu et al. 2016 , where referents (objects in images) are assumed to be deterministically grounded once specified. Some recent work shows that VLMs struggle to adopt different frames of reference and visual perspectives Zhang et al. 2024 ; Lee et al. 2025 . Moreover, existing benchmarks rarely evaluate whether models can track evolving, asymmetric interpretations as dialogue progresses. Our work addresses this gap by annotating personal

Read the complete text