Decision-Level Fusion for Robust Wearable Affect Recognition Conference: In 7th International Workshop on Agents for Societal Impact (ASI 2026) @ AAMAS 2026; May 25, 2026; Paphos, Cyprus <<submission id>>
Lokesh Singh, Athina Georgara, Jayati Deshmukh, Tan Viet Tuyen Nguyen, Sarvapali D. Ramchurn · 2026
Read Decision-Level Fusion for Robust Wearable Affect Recognition Conference: In 7th International Workshop on Agents for Societal Impact (ASI 2026) @ AAMAS 2026; May 25, 2026; Paphos, Cyprus <<submission id>> by Lokesh Singh, Athina Georgara, Jayati Deshmukh, Tan Viet Tuyen Nguyen, Sarvapali D. Ramchurn (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-14878.txt.
Introduction
Singh and team study decision fusion for robust wearable affect detection. Body signals promise public health and stress-aware help, but deployments face shifting dynamics, artefacts, and missing sensors. Fixed-basis spectral features can oversmooth short-lived patterns. The authors combine Fourier-Bessel Series Expansion with EWT spectral segmentation for mode-wise transient descriptors. Signals are weighted by predictive uncertainty and reliability. On WESAD, aggregation beats or matches feature-level aggregation about eighty-four percent of the time.
Catalog record
| CREATOR | Lokesh Singh, Athina Georgara, Jayati Deshmukh, Tan Viet Tuyen Nguyen, Sarvapali D. Ramchurn |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 22107 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.14878
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Decision-Level Fusion for Robust Wearable Affect Recognition Conference: In 7th International Workshop on Agents for Societal Impact (ASI 2026) @ AAMAS 2026; May 25, 2026; Paphos, Cyprus <<submission id>> Lokesh Singh, Athina Georgara, Jayati Deshmukh, Tan Viet Tuyen Nguyen, Sarvapali D. Ramchurn Abstract . Automatic recognition of affective state from wearable physiology has clear societal impact for public health, preventive care, and stress-aware interventions, but real deployments require robustness to non-stationary dynamics, artefacts, and missing sensors. We study this problem on WESAD (baseline, stress, amusement), where common fixed-basis spectral features (e.g., FFT bandpower and Welch PSD) can oversmooth short-lived discriminative patterns. We propose a non-stationary pipeline that combines Fourier–Bessel Series Expansion (FBSE) with EWT data driven spectral segmentation to extract mode wise transient descriptors. For multimodal integration, we adopt decision-level aggregation over per-modality predictors and weight each modality by predictive uncertainty and modality reliability. On WESAD (15 subjects; ECG, EDA, BVP, EMG, ACC three classes) indicate that decision-level aggregation is ≈ 84 % \approx 84\% times at least as good as feature level aggregation, and ≈ 48 % \approx 48\% strictly better, suggesting improved robustness under heterogeneous and partially reliable sensing. C. Amato, L. Dennis, V. Mascardi, J. Thangarajah (eds.) 1. Introduction Automatic affective state detection (a person’s current subjective, psychological, and physiological state) from physiological signals is a key problem in AI for health, with applications such as continuous mental-health monitoring, stress-aware interventions, and adaptive human–computer interaction ( 9 ; 11 ) . Among affective states, stress and amusement capture complementary high-arousal conditions and enable continuous assessment of emotional well-being using wearable sensing in naturalistic settings ( 13 ) . Recent advances in wearable technology make it possible to collect multimodal signals such as electrodermal activity (EDA), electrocardiography (ECG), electromyography (EMG), blood volume pulse (BVP), and motion (ACC), which reflect autonomic and somatic correlates of affect. Benchmark datasets such as WESAD facilitate systematic evaluation of stress and affect recognition under standard protocols, accelerating progress toward deployable affect-aware health systems ( 14 ) . Despite this progress, accurate detection remains challenging because physiological time series are noisy, non-stationary, and subject-dependent, and affect-related dynamics often appear as transient, context-dependent changes rather than stable periodic patterns. A major limitation of many pipelines is feature extraction. Common approaches rely on statistical summaries or Fourier-based representations computed over fixed windows (e.g., FFT bandpower or Welch PSD). These methods implicitly assume approximate local stationarity and can discard time-varying structure that is critical for distinguishing affective state ( 10 ) . Time–frequency methods such as STFT and wavelets partially mitigate this limitation, but they retain trade-offs between time resolution, frequency resolution, and adaptability to signal-specific structure ( 16 ) . Motivated by these limitations, we focus on non-stationarity decomposition methods and, in particular, we propose the FBSE-EWT signal analysis method, which combines Fourier Bessel-series expansion with Empirical Wavelet Transform like data-driven segmentation to better preserve transient spectral structure in physiological windows ( 7 ) . Beyond limitations in physiological signal analysis, wearable affect monitoring highly relies on multimodal sensing to leverage complementary physiological mechanisms. Combining various physiological signals (such as ECG, EMG, EDA, etc.) using machine learning has proven to improve the affective state detection ( 3 ) . However, this introduces a second major challenge: the robust integration under sensor noise, signal failure, and missing modalities, common issues in real-life deployments ( 8 ) . Early fusion ( 5 ) approaches and models that assume complete sensor availability can therefore fail under realistic conditions. To address this challenge, we adopt a modular approach in which each modality computes an independent prediction, followed by decision-level aggregation that accounts for predictive uncertainty. This design is suitable for agent-based decision support because it outputs calibrated belief states that can be used to trigger interventions conservatively under uncertainty. In summary, we propose a three-stage pipeline for affective state detection from wearable physiology. First, we extract meaningful representations using feature extraction techniques tailored to non-stationary time series. Second, we train single-sensor predictors to classify sensor-specific features into affective state, enabling each modality to independently learn affect-discriminative patterns. Finally, we aggregate the multiple sensor classifiers to reach one final prediction. We do so by weighting each sensor’s contribution according to its predictive uncertainty in order to enhance robustness to sensor variability and partial failure. We evaluate the proposed framework on WESAD for three-class affect recognition (baseline / stress/ amusement). Our results suggest that non-stationary feature extraction and decision-level aggregation improve reliability in multimodal settings. The key contributions of this paper are as follows: (1) A non-stationary representation for wearable physiology based on FBSE–EWT that preserves transient spectral structure via data-driven mode isolation. (2) A modular per-modality prediction design and an uncertainty-aware, entropy-weighted decision fusion rule that remains stable under noisy or missing modalities. (3) An empirical evaluation on WESAD demonstrating the benefit of decision-level aggregation across sensors. S_{1} S_{2} S_{3} S_{4} R_{1} \vdots S_{i} S_{j} R_{m} S_{n} Classifier Classifier Classifier Classifier Classifier Team Formatidon Aggregation Affective state Decision Final Class Prediction Fusion Module Multimodal Affective State Identification Figure 1 . Wearable affect recognition flow: windowing, FBSE–EWT features, per-modality prediction, and uncertainty-weighted decision fusion. 2. Related Work 2.1. Wearable affect detection Wearable affect recognition is commonly framed as supervised classification from multimodal physiological Signals, with WESAD dataset becoming a standard benchmark because it provides synchronized chest and wrist modalities and three affective conditions (baseline, stress, amusement) ( 14 ) . Recent work explores both classical Machine learning and deep learning approaches. CNN/LSTM-attention style models are frequently reported as strong baselines for multimodal WESAD classification ( 15 ) , reflecting the value of temporal modelling when affect clues over time. However, cross-subject generalization remains difficult due to subject-dependent physiology and non-stationary responses. Many methods on WESAD still depends on representation that are either overly aggregated (loosing transient nature) or not designed to be useful across modalities and subjects, leaving space for representation learning that better matches the signal properties of wearable physiology. 2.2. Frequency and time-frequency representation A major line of work relies on frequency domain coefficients, Fourier-domain descriptors and bandpower features are widely used due to simplicity and interpretability. Welch’s method is a standard PSD estimator that reduces variance relative to raw FFT periodograms through averaging of modified periodogram ( 17 ) . These estimators are effective when signals are approximately stationary within short windows, but physiological affect responses can be non-stationary and event-like, where averaging may smooth discriminative structure. Time-frequnecy methods partially address this limitation. STFT provides a localised spectrum but requires a fixed window and thus faces inherent time–frequency resolution trade-offs ( 2 ) . Wavelet decompositions (e.g., DWT) provide multi-resolution analysis and are widely used for physiological signals, but still rely on pre-defined basis structure rather than fully data-adaptive segmentation ( 12 ) . Fixed basic spectral and time frequency methods impose analysis choices (band, windows, wavelet families) that can underfit the transient and signal specific structure present in wearable affect data, motivation more adaptive decomposition method that followed the observed spectrum. 2.3. Adaptive decompositions for non-stationary physiological signals To better match non-stationary physiology, research has increasingly used adaptive decompositions such as empirical mode decomposition (EMD/HHT), Varitonal mode decomposition (VMD). EMD decomposes a signal into intrinsic mode functions that