Leveraging Large Language Models to Identify Conversation Threads for Collaborative Learning Analysis
Prerna Ravi* MIT CSAIL Cambridge, MA, USA [email protected], Dong Won Lee* MIT Media Lab Cambridge, MA, USA [email protected], Beatriz Flamia Instituto Politécnico de Bragança Bragança, Portugal [email protected], Jasmine David MIT STEP Cambridge, MA, USA [email protected], Brandon Hanks MIT STE · 2025
Read Leveraging Large Language Models to Identify Conversation Threads for Collaborative Learning Analysis by Prerna Ravi* MIT CSAIL Cambridge, MA, USA [email protected], Dong Won Lee* MIT Media Lab Cambridge, MA, USA [email protected], Beatriz Flamia Instituto Politécnico de Bragança Bragança, Portugal [email protected], Jasmine David MIT STEP Cambridge, MA, USA [email protected], Brandon Hanks MIT STE (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-22844.txt.
Introduction
Group talk weaves several topic threads at once. Tracing those threads in live speech is hard. Prerna Ravi leads a team with seven co-authors. In 2025 they test language models on this task. Their setting is collaborative learning in small groups.
The team wrote a guidebook for finding threads. It covers synchronous multi party transcripts. They benchmark prompts for automated threading. Then they test downstream coding of talk moves. Moves include agreeing, building, and eliciting. Clear thread data improved model coding scores. The work links discourse structure to learning analysis. Overlapping turns and implicit cues remain core challenges.
Catalog record
| CREATOR | Prerna Ravi* MIT CSAIL Cambridge, MA, USA [email protected], Dong Won Lee* MIT Media Lab Cambridge, MA, USA [email protected], Beatriz Flamia Instituto Politécnico de Bragança Bragança, Portugal [email protected], Jasmine David MIT STEP Cambridge, MA, USA [email protected], Brandon Hanks MIT STE |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 100 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2510-22844
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Leveraging Large Language Models to Identify Conversation Threads for Collaborative Learning Analysis Prerna Ravi* MIT CSAIL Cambridge, MA, USA [email protected], Dong Won Lee* MIT Media Lab Cambridge, MA, USA [email protected], Beatriz Flamia Instituto Politécnico de Bragança Bragança, Portugal [email protected], Jasmine David MIT STEP Cambridge, MA, USA [email protected], Brandon Hanks MIT STEP Cambridge, MA, USA [email protected], Cynthia Breazeal MIT Media Lab Cambridge, MA, USA [email protected], Emma Anderson MIT STEP Cambridge, MA, USA [email protected], Grace Lin MIT STEP Cambridge, MA, USA [email protected] Abstract Understanding how ideas develop and flow in small-group conversations is critical for analyzing collaborative learning. A key structural feature of these interactions is threading—the way discourse talk naturally organizes into interwoven topical strands that evolve over time. While threading has been widely studied in asynchronous text settings, detecting threads in synchronous spoken dialogue remains challenging due to overlapping turns and implicit cues. At the same time, large language models (LLMs) show promise for automating discourse analysis but often struggle with long-context tasks that depend on tracing these conversational links. In this paper, we investigate whether explicit thread linkages can improve LLM-based coding of relational moves in group talk. We contribute a systematic guidebook for identifying threads in synchronous multi-party transcripts and benchmark different LLM prompting strategies for automated threading. We then test how threading influences performance on downstream coding of conversational analysis frameworks, that capture core collaborative actions such as agreeing, building, and eliciting. Our results show that providing clear conversational thread information improves LLM coding performance and underscores the heavy reliance of downstream analysis on well-structured dialogue. We also discuss practical trade-offs in time and cost, emphasizing where human-AI hybrid approaches can yield the best value. Together, this work advances methods for combining LLMs and robust conversational thread structures to make sense of complex, real-time group interactions. Keywords: Threading, Conversational Analysis, Large Language Models (LLMs), Collaborative Learning 1 Introduction Analyzing how small groups collaborate through talk is a cornerstone of research in learning sciences, organizational behavior, and computational discourse analysis ( Reyes & Tchounikine (2003) ; Bales (1950) ; Armstrong & Priola (2001) ; Hennessy et al. (2016) ; Cakir et al. (2005) ) . Decades of work have shown that the ways participants interact—how they build on, challenge, or elicit ideas—profoundly shape team outcomes, learning, and satisfaction ( E.E. Smith & Kight (1959) ; Kim et al. (2008) ) . However, tracing these interaction patterns reliably remains a persistent challenge, especially in synchronous spoken setting s where conversational dynamics are fluid, implicit, and overlapping ( McDaniel et al. (1996) ) . A core structural feature of collaborative dialogue is threading , the idea that conversations naturally organize into interleaved topical strands that develop, branch, and reconnect over time ( McDaniel et al. (1996) ; Ji et al. (2021) ) . Identifying these threads is vital for making sense of how ideas flow, how meaning is negotiated, and how groups reach shared understanding ( Reyes & Tchounikine (2003) ) . While threading is well-supported in asynchronous contexts like online forums—where reply structures are explicit and easily reviewable ( Clark & Brennan (1991) ) — its detection in real-time, multi-party speech is far more complex ( Egbert (1997) ) . Prior work has highlighted that without reliable threading, important links between contributions can be lost, limiting the value of downstream conversational analysis frameworks that depend on these connections ( Chang et al. (2023) ; Herring (1999) ) . In parallel, the rise of large language models (LLMs) has opened up new opportunities for automating conversation analysis ( Lee et al. (2025) ; Martinenghi et al. (2024) ) . Yet, recent studies show that LLMs often struggle with tasks that require tracking long-range discourse dependencies ( Hsieh et al. (2024) ; Liu et al. (2024) ) , such as classifying dialogue acts in extended transcripts. This raises a critical question: can explicit threading—when supplied to or generated by LLMs—improve the consistency and accuracy of their downstream coding? This paper tackles this question in the context of collaborative learning analysis, using an example framework from Lin et al. (2025) . We investigate how well LLMs can identify threads in synchronous small-group transcripts and whether providing explicit thread linkages—either human-annotated or LLM-generated—boosts the quality of relational coding for elicitation moves. In doing so, we bridge methodological gaps in discourse research with practical questions of cost, reliability, and human-AI complementarity. Our contributions are threefold: (1) We develop and test an operational definition and guidebook for threading in synchronous small-group talk, grounded in an iterative human annotation process, (2) We benchmark multiple LLM prompting strategies for automated threading, (3) We empirically test how threading impacts LLM performance on downstream coding of conversational analysis frameworks, providing new insights into where threads help, where errors persist, and what this means for scaling reliable collaborative discourse analysis. Together, this work advances practical strategies for researchers and educators who want to use LLMs to make sense of complex group interactions while showing that better threading is not just a technical pre-processing step, but a critical scaffold for deeper conversational understanding. 2 Related works 2.1 Understanding Interaction Patterns in Synchronous Collaboration Understanding how groups collaborate remains critical for advancing theories of collective work and for designing systems that support such interactions. Research has underscored that patterns of communication are among the strongest predictors of a work team’s success and are as influential as individual intelligence, personality, skill, or even the substantive content of discussions combined ( Pentland (2012) ) . In particular, these communication patterns can reliably distinguish high-performing teams ( Pentland (2012) ) . Scholars have also emphasized the importance of analyzing these fine-grained interaction patterns in the context of collaborative learning ( Stahl (2006) ) . In the education space, conversational analysis provides a powerful lens for examining the social, communicative, and problem-solving interactions that underlie collaborative learning and group performance ( Cakir et al. (2005) ) . Highlighting structures such as turn-taking conventions and adjacency pairs—typical sequences like question-answer or offer-response—helps reveal how participants negotiate meaning, coordinate actions, and maintain coherent dialogue ( M. Smith et al. (2000) ) . These techniques form the backbone of natural spoken interaction ( M. Smith et al. (2000) ) . However, these natural multi-party settings are marked by spontaneous and overlapping talk, disfluencies, and unclear sentence boundaries—features that have presented scholars with significant challenges for analyzing speech over decades ( Krauss et al. (1977) ; Shriberg (2005) ) . To address this, researchers typically start by examining who speaks to whom moment by moment ( Gatica-Perez (2006) ) . Over extended interactions, this helps uncover additional patterns of recurring group dynamics such as floor control, dominance, and collective interest ( Gatica-Perez (2006) ) . Taken together, these insights highlight that making sense of collaborative group talk depends on tracing how ideas and actions connect across turns . We explore this further in the next section through the concept of threading. 2.2 Threading in Synchronous Group Interactions Threading, a core concept in the study of group interactions, refers to how conversations naturally organize into coherent topical strands that develop, diverge, and reconnect over time ( McDaniel et al. (1996) ) . Identifying these threads is vital for understanding how ideas