MIS-LSTM: Multichannel Image–Sequence LSTM for Sleep Quality and Stress Prediction
Seongwan Park, Jieun Woo, Siheon Yang · 2025
Read MIS-LSTM: Multichannel Image–Sequence LSTM for Sleep Quality and Stress Prediction by Seongwan Park, Jieun Woo, Siheon Yang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-11232.txt.
Introduction
Seongwan Park, Jieun Woo, and Siheon Yang present MIS-LSTM in this 2025 health informatics paper. The system predicts a person's sleep quality and stress level for each day from lifelog data, the steady stream of sensor readings and events that wearables and phones collect.
Continuous sensor streams are chopped into blocks of a few hours and drawn as multi-channel images, while occasional discrete events get their own one-dimensional encoder. An attention module blends the two streams, and a recurrent network reads the sequence of blocks to catch patterns across the day. An uncertainty-aware ensemble then overrules shaky majority votes with confident individual ones. On a 2025 lifelog challenge dataset the full system beat strong recurrent and convolutional baselines, and tests confirmed that four-hour blocks and the multi-channel layout each helped.
Catalog record
| CREATOR | Seongwan Park, Jieun Woo, Siheon Yang |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 29 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2509-11232
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
MIS-LSTM: Multichannel Image–Sequence LSTM for Sleep Quality and Stress Prediction Seongwan Park, Jieun Woo, Siheon Yang Abstract This paper presents MIS-LSTM, a hybrid framework that joins CNN encoders with an LSTM sequence model for sleep quality and stress prediction at the day level from multimodal lifelog data. Continuous sensor streams are first partitioned into N-hour blocks and rendered as multi-channel images, while sparse discrete events are encoded with a dedicated 1D-CNN. A Convolutional Block Attention Module fuses the two modalities into refined block embeddings, which an LSTM then aggregates to capture long-range temporal dependencies. To further boost robustness, we introduce UALRE, an uncertainty-aware ensemble that overrides low-confidence majority votes with high-confidence individual predictions. Experiments on the 2025 ETRI Lifelog Challenge dataset show that Our base MIS-LSTM achieves Macro-F1 0.615; with the UALRE ensemble, the score improves to 0.647, outperforming strong LSTM, 1D-CNN, and CNN baselines. Ablations confirm (i) the superiority of multi-channel over stacked-vertical imaging, (ii) the benefit of a 4-hour block granularity, and (iii) the efficacy of modality-specific discrete encoding. 1 Introduction Poor sleep quality and high stress levels are related to serious health consequences, including elevated risks of cardiovascular disease, depression, impaired cognition, and weakened immune function. Chronic sleep deprivation also disrupts emotional regulation, creating a vicious cycle between stress and sleep problems [ 1 , 2 ] . These issues have driven growing interest in lifelog data from wearables and smartphones as a means to continuously monitor daily behaviors and infer sleep and mental well-being. Lifelog sensor streams (e.g., activity, heart rate) provide rich information reflecting an individual’s daily lifestyle, which can yield insights into sleep patterns and stress in naturalistic settings [ 3 ] . Advanced wearable sensors combined with deep learning models enable unobtrusive health monitoring from such data [ 4 ] , and have already shown promise in automatic sleep-stage recognition [ 5 ] . Figure 1: Overview of a Multichannel Image–Sequence LSTM (MIS-LSTM) framework (4-hour block intervals). Recent advances in time-series modeling fall into two complementary paradigms. On one hand, sequential architectures—such as Long Short-Term Memory networks (LSTMs) and Time-Series Transformers (TSTs) [ 6 ] , which—being inspired by the Transformer architecture [ 7 ] —capture long-range sequences. However, their emphasis on global sequence modeling often comes at the expense of fine-grained, non-linear motifs that manifest over short intervals. On the other hand, image-based approaches embed fixed-length windows of the series as two-dimensional “images” and apply convolutional neural networks (CNNs). Through sliding convolutional filters, CNNs are well-suited to detecting localized temporal patterns—such as periodic cycles, abrupt transitions, and peaks—by convolving over adjacent time steps [ 8 ] . Moreover, by allocating each continuous variable to its own channel, these models learn cross-channel interactions via 3D kernel (C×H×W), preserving cross-feature interactions while encoding each modality efficiently. However, despite their powerful local feature extraction, CNNs’ fixed receptive fields and the implicit loss of exact temporal ordering limit their ability to reason over long-range temporal dependencies. To integrate the complementary strengths of convolutional and LSTM architectures while addressing their respective shortcomings, we introduce MIS-LSTM: Multichannel Image–Sequence LSTM, a hybrid CNN–LSTM architecture. In our day-level sleep quality and stress prediction framework, raw sensor streams are first divided into fixed-length, N-hour blocks . Within each block, we distinguish between two feature types: • Continuous features (e.g., heart rate, distance traveled) are real-valued and dense, capturing fine-grained physiological or motion signals • Discrete features (e.g., activity category, smartphone on/off status) are integer-valued and sparse, encoding categorical or event-driven states. Because these modalities exhibit different properties, jointly convolving them can impede learning; instead, each is processed through a dedicated CNN encoder optimized for its characteristics. The block-wise feature maps produced by these encoders are then concatenated and passed through a Convolutional Block Attention Module (CBAM) [ 9 ] to emphasize salient patterns across feature maps, yielding a refined N-hour representation. Finally, the sequence of these block representations serves as input to a LSTM, which captures global, day-level temporal dependencies. This two-stage pipeline (i) excels at extracting localized, nonlinear motifs via CNNs and (ii) integrates these local features over longer horizons through the LSTM, leading to superior performance on lifelog-based time-series tasks. Our method is summarized in Figure 1 . Furthermore, we propose a novel ensemble strategy that improves robustness by leveraging prediction confidence. Multiple model variants are trained, and for each sample, we examine the logit margin (difference between the top two or three class scores) as a confidence measure. For ambiguous cases (low margin), standard majority voting is used, whereas for certain cases (high margin) where the top-performing model strongly disagrees with the ensemble majority, we selectively override the majority vote with the confident prediction. This adaptive voting mechanism systematically resolves conflicting votes by privileging the most confident model, effectively correcting cases where the majority might be mistaken. Experimental results show that on a public lifelog dataset from the 2025 challenge (predicting daily sleep quality (Q1–Q3) and sleep health metrics (S1–S3)), we demonstrate that the proposed framework significantly outperforms baselines. In particular, it yields substantially higher Macro-F1 scores than: (a) an raw LSTM baseline, (b) a raw CNN and only use 1D-CNN, (c) our model variant in which the dedicated discrete feature branch is omitted. We also conduct ablation studies to validate the benefit of multi-channel image encoding over alternative representations (tested on N-hour blocks with N in 2,4,6), and ensemble refinement on performance. The contributions of this paper are as follows: 1. Hybrid Sequential–Image Framework: We introduce a two-stage CNN– LSTM architecture for time-series prediction that encodes time-series into N-hour block representations for local pattern extraction via CNN, and integrates these representations across N-hour blocks with an LSTM to capture long-range temporal dependencies. 2. Modality-Specific Feature Modeling: We design a hybrid processing pipeline that preserves feature characteristics by handling continuous sensor streams through multi-channel CNN encoding and discrete, sparse signals through a dedicated 1D-CNN branch, yielding superior performance compared to merging all features in a single representation. 3. Uncertainty-Aware Logit-Refined Ensemble (UALRE): We propose a novel ensemble strategy that computes logit margins as a confidence signal, applying standard majority voting for low-confidence samples and selectively overriding the vote when a highly confident model conflicts with the ensemble, thereby improving robustness over traditional soft or hard voting schemes. 2 Related Work Table 1: Overview of multimodal lifelog sensor streams, including frequency (Hz.), data types, and brief descriptions of each feature. Items Freq. Data type Note mACStatus 1/60 integer 0: No, 1: Charging mActivity 1/60 integer 0: in_vehicle, 1: on_bicycle, 2: on_foot, 3: still, 4: unknown, 5: tilting, 7: walking, 8: running mAmbience 1/120 object List of ambient sound labels and their probabilities mBle 1/600 object List of Bluetooth device address, device_class, and RSSI mGps 1/60 object List of (altitude, latitude, longitude, speed) mLight 1/600 float Ambient light in lx unit mScreenStatus 1/60 integer 0: No, 1: Using screen mUsageStats 1/600 object List of app names and their usage times (ms) mWifi 1/600 object List of base station ID (BSSID) and RSSI wHr 1/60 object List of heart rate recordings wLight 1/600 float Ambient light in lx unit wPedo 1/60 float; float; float; integer; float Number of calories; Distance in meters; Speed in km/h unit; Number of steps; Step frequency