Inclusive AI for Group Interactions: Predicting Gaze-Direction Behaviors in People with Intellectual and Developmental Disabilities
Giulia Huang, Maristella Matera, Micol Spitale · 2026
Read Inclusive AI for Group Interactions: Predicting Gaze-Direction Behaviors in People with Intellectual and Developmental Disabilities by Giulia Huang, Maristella Matera, Micol Spitale (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-14460.txt.
Introduction
Huang, Matera and Spitale build tools to read gaze in group talks with disabled participants. Support agents struggle when users are not neurotypical. Most detectors train on neurotypical data and miss atypical gaze cues. The team releases MIDD, a dataset of multi-party talks with people with intellectual and developmental disabilities. They compare it with neurotypical sets and test classifiers from SVMs to FSFNet. Fine-tuning on MIDD helps, the authors find, though clear limits remain. Six therapists add context in a focus group.
Catalog record
| CREATOR | Giulia Huang, Maristella Matera, Micol Spitale |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 44275 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.14460
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Inclusive AI for Group Interactions: Predicting Gaze-Direction Behaviors in People with Intellectual and Developmental Disabilities Giulia Huang, Maristella Matera, Micol Spitale Abstract Artificial agents that support human group interactions hold great promise, especially in sensitive contexts such as well-being promotion and therapeutic interventions. However, current systems struggle to mediate group interactions involving people who are not neurotypical. This limitation arises because most AI detection models (e.g., for turn-taking) are trained on data from neurotypical populations. This work takes a step toward inclusive AI by addressing the challenge of eye contact detection, a core component of non-verbal communication, with and for people with Intellectual and Developmental Disabilities. First, we introduce a new dataset, Multi-party Interaction with Intellectual and Developmental Disabilities (MIDD), capturing atypical gaze and engagement patterns. Second, we present the results of a comparative analysis with neurotypical datasets, highlighting differences in class imbalance, speaking activity, gaze distribution, and interaction dynamics. Then, we evaluate classifiers ranging from SVMs to FSFNet, showing that fine-tuning on MIDD improves performance, though notable limitations remain. Finally, we present the insights gathered through a focus group with six therapists to interpret our quantitative findings and understand the practical implications of atypical gaze and engagement patterns. Based on these results, we discuss data-driven strategies and emphasize the importance of feature choice for building more inclusive human-centered tools. I Introduction Artificial agents that support human group interactions hold great promise, particularly in sensitive contexts such as well-being promotion [ 24 , 12 ] and therapeutic interventions [ 7 ] . By helping to structure conversations [ 8 ] , encourage participation [ 10 ] , or mediate turn-taking [ 11 ] , these systems can enhance social engagement in group settings. However, most current systems remain narrowly designed for neurotypical populations [ 28 ] . This limitation stems from the training data: nearly all existing AI detection models that detect speaking turns, gaze direction, or attention, are built on corpora reflecting the behaviors of neurotypical participants, which hinders their ability to generalize to heterogeneous and inclusive group settings. Eye contact, in particular, plays a central role in multi-party social interaction [ 19 ] , yet the vast majority of AI systems that detect it are trained on datasets reflecting standard social dynamics among neurotypical participants, leading to systematic failures in inclusive contexts. This issue is particularly evident in conversations involving individuals with Intellectual and developmental disabilities (IDDs), lifelong conditions affecting cognitive and adaptive functioning [ 21 ] . Individuals with IDD often exhibit atypical interaction patterns, including reduced or indirect gaze, alternative non-verbal cues (e.g., gestures or vocalizations), and turn-taking rhythms that diverge from neurotypical norms. Moreover, their interactions are frequently mediated by external scaffolds such as caregiver prompts or augmentative communication devices [ 22 ] , which are absent from conventional training data. As a result, AI models for gaze and turn-taking not only lose accuracy but risk marginalizing interaction styles that deviate from the average user. Positioned within the field of human-centered and inclusive AI [ 6 ] , this paper investigates group dialogues involving participants with IDD. We introduce the Multi-party Interaction with Intellectual and Developmental Disabilities (MIDD) dataset, a carefully annotated corpus that mirrors existing benchmarks such as the MultiMediate dataset [ 17 ] , while also capturing the rich behavioral diversity of IDD populations. Building on this resource, we analyze the distributional shifts that distinguish neurodiverse from neurotypical interactions. To evaluate the impact of these shifts, we benchmark a range of eye-contact detectors, from classical approaches such as Support Vector Classifier (SVC) and eXtreme Gradient Boosting (XGBoost), to the state-of-the-art deep learning model Feature Selection and Fusion Network (FSFNet) [ 14 ] . We assess both zero-shot performance and fine-tuning on MIDD, highlighting improvements as well as persistent limitations. Finally, we involved six therapists in a focus group to better contextualize our quantitative findings and gather expert insights on interpreting atypical gaze and engagement behaviors. By documenting behavioral differences (e.g., class imbalance, speaking activity, gaze distribution) and evaluating model robustness, this work contributes empirical insights and a practical foundation for developing inclusive, multimodal eye-contact systems that promote fairness and accessibility in real-world human–computer interaction. The contributions of this paper are the following: • We introduce a novel IDD-focused dataset (anonymized features are openly available 1 1 1 https://github.com/giuliahuang/EyeContactModels/tree/main/Dataset ) designed for multi-party group conversations, mirroring the MultiMediate benchmark while capturing unique neurodiverse interaction patterns. • We conduct a rigorous comparison analysis of MIDD with existing datasets (of neurotypical populations), highlighting differences in behaviours , namely gaze behavior, speaking activity, interaction dynamics, and recording quality. • We evaluate performances of both classical (SVC, XGBoost) and state-of-the-art (FSFNet) models , showing their limitations in generalising to IDD populations and the improvements gained via fine-tuning. • We incorporate expert insights from six therapists via a focus group, contextualizing quantitative results and informing the design of inclusive, human-centered interaction systems. This paper provides insights for inclusive AI by demonstrating the need for multimodal features (e.g., gaze, head pose) and domain adaptation strategies to accurately model turn-taking and eye-contact in diverse group interactions. II Related Works II-A Individuals with IDD in group interactions Prior research shows that individuals with intellectual and developmental disabilities (IDD) often struggle to claim and maintain turns in multi-party conversations [ 5 ] . Cognitive–linguistic constraints, atypical gaze patterns, and reliance on augmentative communication can reduce verbal initiations and eye contact, while caregivers may unintentionally dominate the dialogue [ 3 , 27 ] . Consequently, adults with IDD are at increased risk of marginalization or passive participation in both informal and structured group settings. These difficulties are observed across diverse IDD profiles, including syndromic IDD (e.g., Down syndrome, autism spectrum disorder), neurological impairments, and co-occurring mental health conditions, which can further disrupt conversational coordination. Supporting equitable turn-taking for individuals with IDD is therefore a crucial issue for promoting inclusion and social participation. These challenges highlight the potential of AI-driven artificial agents to support and moderate group interactions more efficiently. By leveraging real-time detection of gaze, attention, and turn-taking patterns, AI models can guide conversation flow, provide adaptive prompts, promote more balanced participation in multi-party settings. II-B Multi-party turn-taking detection Past research has tackled turn-taking in conversations using diverse modalities. Early models focused on textual and syntactic features, e.g., TurnGPT and CNN-FSA hybrids, while later work incorporated prosodic and audio cues (e.g., pitch, speed, volume), yet lacked the ability to infer addressees, limiting their utility in multi-party settings. Visual cues, particularly eye-gaze, have emerged as powerful signals for coordinating turn transitions and managing conversational flow. Recent multimodal approaches combine audio, visual, and gestural information, demonstrating improved performance in predicting turn-taking behavior, as exemplified by benchmarks such as the MultiMediate Challenge [ 17 ] . Müller et al. [ 18 ] further showed that eye-contact detectors can be trained using weak supervision from speaker behavior, enabling unobtrusive gaze modeling. To push state-of-the-art further, FSFNet by Ma et al. [ 14 ] combines CNNs with Transformers and adaptive feature selection to robustly detect eye contact under challenging visual conditions. Despite these advances, all benchmarks so far are built on neurotypical populations. No existing work has studied eye-contact behavior in people with Intellectual and Developmental Disabilities (IDD), despite