AffectAI-Capture: A Reproducible Multimodal Protocol for Small-Group Meeting Research

Meisam Jamshidi Seikavandi, Alice Modica, Anna Obara, Fabricio Batista Narcizo, Tanya Ignatenko, Ted Vucurevich, Jesper Bünsow Boldt, Paolo Burelli, Andrew Burke Dittberner · 2026

Read AffectAI-Capture: A Reproducible Multimodal Protocol for Small-Group Meeting Research by Meisam Jamshidi Seikavandi, Alice Modica, Anna Obara, Fabricio Batista Narcizo, Tanya Ignatenko, Ted Vucurevich, Jesper Bünsow Boldt, Paolo Burelli, Andrew Burke Dittberner (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-19794.txt.

Introduction

A reproducible protocol for four-person meeting capture comes from Seikavandi and colleagues in AffectAI-Capture. Eye tracking, wearable body sensors, room and close audio, multi-view video, event logs and self-reports share one timeline with standard outputs. Bench tests check audio quality and video timing, while full sessions with people remain in progress. The work links task design, tools, timing records and packaging for meeting research.

Catalog record

CREATORMeisam Jamshidi Seikavandi, Alice Modica, Anna Obara, Fabricio Batista Narcizo, Tanya Ignatenko, Ted Vucurevich, Jesper Bünsow Boldt, Paolo Burelli, Andrew Burke Dittberner
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE20639
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.19794

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

AffectAI-Capture: A Reproducible Multimodal Protocol for Small-Group Meeting Research Thanks: Accepted at the AI4Action Workshop, 19th International Conference on PErvasive Technologies Related to Assistive Environments (PETRA 2026). Published by Springer in Communications in Computer and Information Science (CCIS). Meisam Jamshidi Seikavandi, Alice Modica, Anna Obara, Fabricio Batista Narcizo, Tanya Ignatenko, Ted Vucurevich, Jesper Bünsow Boldt, Paolo Burelli, Andrew Burke Dittberner Abstract We present AffectAI-Capture, a protocol for collecting synchronized multimodal data in four-person meeting-like interactions, combining eye tracking, wearable physiology, close-talk and room audio, multi-view video, event logging, and structured self-report. Sessions use fixed task blocks grounded in established group-interaction paradigms, while acquisition and post-processing are organized around a single authoritative event timeline and standardized outputs. We describe the experimental rationale, synchronization philosophy, data organization, and practical trade-offs. Pilot-level validation of audio quality and video synchronization has been conducted using controlled bench tests; full protocol sessions with participants remain ongoing work. The contribution is a reproducible protocol architecture linking task design, instrumentation, timing provenance, and data packaging for affective, behavioral, and meeting-analytics research. 1 Introduction Understanding group interaction in realistic meetings requires more than recording speech and video. Researchers increasingly seek to study how attention, arousal, effort, coordination, and subjective experience unfold jointly across participants and over time, demanding multimodal datasets that combine rich sensing, interpretable task structure, auditable synchronization, and reusable data organization. Classic meeting corpora showed the importance of synchronized audio-visual capture and structured meeting scenarios for cross-session comparability [ 5 , 9 ] . More recent affective and physiological datasets demonstrated the value of eye-related and wearable measurements for studying internal-state variation, but are rarely built around true small-group meeting interactions [ 13 ] . This leaves a gap between meeting corpora optimized for collaborative structure and affective-computing datasets optimized for participant-level sensing. AffectAI-Capture closes this gap by combining scenario-based small-group tasks with high-resolution multimodal sensing, explicit event-centered synchronization, and BIDS-oriented derived outputs. It builds on our prior work on gaze-based emotion perception, multimodal affect modelling, and the interplay of eye tracking, personality, and temporal dynamics in face-to-face settings [ 20 , 19 , 21 ] , which highlighted the need for richer interaction-centered datasets. The protocol serves as a reusable foundation for affective computing, meeting analytics, and human-centered AI research. 2 Design Goals The protocol is guided by four design goals. 1. Cross-session comparability without abandoning meeting realism. Sessions are structured using fixed task blocks that reproduce distinct social-cognitive interaction regimes, while preserving a plausible overall meeting narrative. 2. Multimodal capture at both group and individual levels. The protocol combines room-level behavioral signals with participant-specific channels such as gaze, pupillometry, skin conductance, and heart rate. 3. Auditable temporal provenance. Synchronization is treated as a first-class design problem. The protocol does not assume perfect device synchronization, but instead records sufficient timing anchors to estimate, validate, and document alignment. 4. Reproducible and reusable outputs. Vendor raw data are preserved, derived outputs are separated, and session outputs are structured in a BIDS-oriented manner to support downstream analysis and long-term reuse [ 8 , 15 , 26 ] . 3 Related Work Classic meeting corpora demonstrated the value of instrumented multi-party data, including structured scenarios and close-talk audio for transcription, overlap handling, and speaker-specific analysis [ 5 , 9 ] . However, these corpora focused on speech, video, and interaction annotation with limited access to participant-level measures such as gaze or physiology. Multimodal affect datasets have shown the value of combining physiology, video, and self-report, but their setups often fell outside genuine small-group collaborative interaction [ 13 ] . K-EmoCon [ 14 ] demonstrated the feasibility of combining wearable physiology and continuous annotation in naturalistic dyadic conversation, and AFFEC [ 23 ] introduced a multimodal dataset for face-to-face emotion communication integrating eye tracking, physiology, and video. Both bridge affective sensing and social interaction more closely than stimulus-response paradigms, but neither targets multi-party meeting settings with structured task variation. AffectAI-Capture is designed at the intersection of these traditions. 4 Experimental Design Session structure. The protocol targets four-participant sessions organized into five blocks: T0 (onboarding/warm-up), T1 (hidden-profile decision), T2 (negotiation), T3 (idea generation and selection), and T4 (public-goods micro-game). The fixed order supports cross-session comparability while progressing through distinct demands: acclimatization, information pooling, strategic exchange, collaborative creativity, and cooperation under mixed incentives. Structured tasks are a scientific asset: they create repeated interaction contexts with known behavioral pressures, making multimodal measurements more interpretable [ 5 ] . Task-specific rationale. Each task block is included because it elicits a distinct and well-studied interaction regime. T1: Hidden-profile decision. This task targets information pooling under asymmetrically distributed knowledge. It is suitable for studying whether groups disclose and integrate unique information, how attention and speaking patterns shift when new evidence is introduced, and how collective decisions are shaped by discussion structure [ 24 , 11 ] . T2: Negotiation. Negotiation introduces strategic interaction, emotional signaling, concession dynamics, and interpersonal regulation. It is well suited for multimodal capture, as vocal, facial, physiological, and gaze-related signals may reflect tension, control, responsiveness, and the evolution of bargaining positions [ 25 ] . T3: Idea generation and selection. This task separates divergent and convergent collaboration. It allows examination of how ideas are generated, shared, compared, and evaluated, as well as how groups converge on a final choice while weighing priorities such as originality, feasibility, or overall appeal [ 17 ] . T4: Public-goods micro-game. This task introduces a controlled tension between private incentives and collective outcomes. It produces clear event boundaries for decision, reveal, and discussion, making it useful for event-locked multimodal analysis of cooperation, surprise, disappointment, and the negotiation of fairness and social norms [ 7 , 6 ] . Task timing. Timed and moderator-controlled phases are intentionally mixed. Timed phases ensure comparability; controlled briefing phases preserve flexibility. Table 1: Task and timing structure. Task Objective Timed phases T0 onboarding free talk: 300 s T1 hidden-profile consensus reading: 75 s; discussion: 420 s; selection: 60 s T2 negotiation negotiation: 480 s; settlement form: untimed T3 ideation generation: 180 s; board & discussion: 420 s; selection: 60 s T4 public-goods game contribution: 60 s; reveal: 60 s; discussion: 180 s 5 Multimodal Instrumentation Primary devices. The stack includes 4 Tobii Pro Glasses 3 (eye tracking), 4 EmotiBit devices (physiology), 5–7 Jabra cameras, 5 DPA microphones (4 close-talk + 1 room/spare), optional Vicon motion capture, and 4 participant tablets with 1 shared display. Camera layout. The video subsystem balances participant coverage, table-level interaction visibility, and synchronization robustness. The setup uses seven room cameras: four desk-adjacent cameras providing participant-focused lateral views and three overview cameras capturing the room more globally. Most cameras are configured for 1920 \times 1080 capture at 30 fps, with the wide-angle room camera subject to device-dependent effective resolution limits in practice. The layout is organized around a desk-centered spatial reference system. A permanent ChArUco board at the desk center defines the world origin, while desk-edge and participant-linked markers support calibration, participant localization, and later multimodal alignment. Four desk-adjacent cameras are physically mounted upside down to clear desk-edge obstructions and achieve downward-angled coverage of the participant’s face and upper body from a compact mounting position; the resulting image is corrected in software to preserve consistent downstream orientation. Capture settings prioritize

Read the complete text