Assessing the feasibility of Large Language Models for detecting micro-behaviors in team interactions during space missions

Ankush, Projna, Sydney, Suzanne, Theodora · 2025

Read Assessing the feasibility of Large Language Models for detecting micro-behaviors in team interactions during space missions by Ankush, Projna, Sydney, Suzanne, Theodora (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-22679.txt.

Introduction

Raut and four co-authors test whether large language models can spot micro-behaviors in team talk. Their 2025 paper uses transcripts from simulated space missions. Such small acts, like discouraging speech, shape how crews work. The team asks whether models can flag each turn of talk.

Several setups were tried, from zero-shot use to fine-tuning on labeled turns. Smaller encoder models missed the rare acts even when weighted. A fine-tuned Llama model did best, reaching mid-range scores. The authors argue such tools could aid crew training. They also warn that rare but key acts stay hard to catch.

Catalog record

CREATORAnkush, Projna, Sydney, Suzanne, Theodora
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE21 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2506-22679

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Assessing the feasibility of Large Language Models for detecting micro-behaviors in team interactions during space missions Ankush, Projna, Sydney, Suzanne, Theodora Abstract We explore the feasibility of large language models (LLMs) in detecting subtle expressions of micro-behaviors in team conversations using transcripts collected during simulated space missions. Specifically, we examine zero-shot classification, fine-tuning, and paraphrase-augmented fine-tuning with encoder-only sequence classification LLMs, as well as few-shot text generation with decoder-only causal language modeling LLMs, to predict the micro-behavior associated with each conversational turn (i.e., dialogue). Our findings indicate that encoder-only LLMs, such as RoBERTa and DistilBERT, struggled to detect underrepresented micro-behaviors, particularly discouraging speech, even with weighted fine-tuning. In contrast, the instruction fine-tuned version of Llama-3.1, a decoder-only LLM, demonstrated superior performance, with the best models achieving macro F1-scores of 44% for 3-way classification and 68% for binary classification. These results have implications for the development of speech technologies aimed at analyzing team communication dynamics and enhancing training interventions in high-stakes environments such as space missions, particularly in scenarios where text is the only accessible data. Raut Paromita Begerowski Bell Chaspari University of Colorado BoulderUSA National Aeronautics and Space AdministrationUSA keywords large language models, zero-shot classification, in-context learning, quantization, micro-behaviors, teams † † email: {ankush.raut,projna.paromita,theodora.chaspari}@colorado.edu, {sydney.r.begerowski,suzanne.t.bell}@nasa.gov 1 Introduction Given their effectiveness in spoken language understanding (SLU) [ 1 , 2 , 3 ] , competence as few-shot learners [ 4 ] , and reasoning abilities [ 5 ] , large language models (LLMs) can potentially infer obscure meanings and semantics from speech in interpersonal communication settings, such as dyadic or multiparty interactions. This capability enables their application across diverse domains, including mental health assessment, educational tutoring systems, and workplace collaboration analysis, where labeled data is sparse and understanding conversational context and intent is crucial for providing meaningful feedback and support. In teamwork settings, LLMs can serve as support tools to foster team cohesion by enhancing informal knowledge-sharing processes [ 6 ] and improving work efficiency through personalized response generation in professional interactions [ 7 ] . Additionally, LLM-powered chatbots have the potential to act as virtual liaisons, facilitating seamless communication, personalized support, and conflict resolution in workplace teamwork [ 8 ] . Micro-behaviors, defined as momentary, subtle linguistic and paralinguistic indicators of thoughts and feelings toward another team member [ 9 , 10 ] can offer unique insights into operationally-relevant team functioning. Depending on how they are communicated, micro-behaviors can uplift or discourage a team member [ 11 ] . Repeated micro-behaviors can also have lasting effects on team interactions. Identifying these momentary acts within team interactions can enhance our understanding of factors influencing team states and performance. Given their transient nature, manually detecting micro-behaviors is highly time-consuming. Therefore, developing automated models for their detection could advance speech technologies that support teamwork. Here, we examine the feasibility of using LLMs to detect micro-behaviors in team interactions during simulated space missions. We investigate the zero-shot micro-behavior classification ability and the fine-tuning of encoder-only sequence classification LLMs, including the Robustly Optimized Bidirectional Encoder Representations from Transformers (BERT) Pretraining Approach (RoBERTa) and DistilBERT, and compare them with few-shot prompting in the instruction fine-tuned version of a decoder-only causal language modeling LLM, including Llama-3.1. Our analysis includes a 3-way classification task, categorizing each turn as uplifting, discouraging, or neither, as well as a binary classification task distinguishing between uplifting and discouraging turns. Results indicate that LLMs are feasible for this task, with decoder-only causal LLMs outperforming encoder-only models, particularly in detecting underrepresented classes with a significantly higher recall. 2 Prior Work Recent research has explored automated methods for detecting micro-aggressions, a term used for commonplace verbal, behavioral, or environmental slights that communicate hostile, derogatory, or negative attitudes toward members of marginalized groups [ 12 ] . These methods include handcrafted lexicons [ 13 ] , unigrams/bi-grams, and topic models [ 14 ] . Recent studies have also investigated use of LLMs for the same task. Caselli et al.’s study on unsupervised microaggression detection computed the embeddings for a set of manually curated seed words known to exhibit gender and racial bias, then compared those seed words against target tokens in the corresponding embedding space [ 15 ] . Embeddings from encoder-only LLMs such as BERT and RoBERTa have also been evaluated for their ability to detect microaggressions in written text [ 16 ] and scripted conversational speech [ 17 ] . The contributions of this paper compared to prior work are as follows: (1) Beyond micro-aggressions, this study aims to detect both positive (uplifting) and negative (discouraging) micro-behaviors–an area that remains largely unexplored [ 9 ] . Unlike micro-aggressions, which often carry gendered or racial connotations, micro-behaviors can be either positive or negative and are highly context-dependent, making their detection more nuanced. (2) The majority of prior work has focused on written speech rather than oral multiparty communication which is contextually more complex and semantically richer. (3) Prior work on LLMs has examined encoder-only sequence classification with pre-trained word embeddings [ 15 , 16 , 17 ] , while decoder-only models that leverage autoregressive generation and in-context learning have not been explored despite their potential advantages in handling nuanced language and contextual dependencies. Additionally, our work focuses on micro-behavior detection using only text transcripts of conversations, making this task even more challenging, particularly in the absence of audio data, which enhances models’ ability to detect signals corresponding to the expression of micro-behaviors. 3 Data Description Our data came from five teams that participated in a 45-day mission at the Human Exploration Research Analog (HERA) of the U.S. National Aeronautics and Space Administration (NASA). Each team took part in a simulated space expedition, performing a geological exploration scenario of the journey to and from Mars’s moon, Phobos. In this paper, we used data collected from the team interaction battery (TIB) tasks that were performed five times (one pre-mission, four in-mission) per crew. Each TIB task had an average duration of 1.5 hours and included a 45-minute decision-making task, in which teams could only arrive at the correct decision through adequate information sharing and teamwork, and a relational task in which crew members responded to various questions that foster interpersonal relations among crew. Depending on the task, each conversational turn (i.e., dialogue) was characterized into an event type as task-related, relational, or a transition period event (i.e., in-between the two tasks). Micro-behaviors were coded according to an adapted version of Smith & Griffins’s (2022) theoretical framework in terms of Violation (i.e., presence of valenced behavior; uplifting/positive or discouraging/negative), Intensity (i.e., force of behavior in terms of how uplifting or discouraging is the behavior), and Intent (i.e., motive of the behavior in terms of whether it was deliberate or unintentional). This paper examines only the Violation dimension in 13,058 conversational turns: 17.8% uplifting, 3.3% discouraging, 75.76% neither, and 3.14% unlabeled. Figure 1: Illustration of the pre-processing steps for encoder-only sequence classification. Figure 2: Illustration of the instruction prompt creation pipeline for decoder-only causal text generation. 4 Methodology In this section, we describe the encoder-only sequence classification (Section 4.1 ) and the decoder-only causal text generation (Section 4.2 ) methods which are used in the 3-way (i.e., uplifting, discouraging, and neither) and binary (i.e., uplifting and discouraging) micro-behavior classification

Read the complete text