Two-Stream Thermal Imaging Fusion for Enhanced Time of Birth Detection in Neonatal Care

Jorge García-Torres 1 , Øyvind Meinich-Bache 1 2 , Sara Brunner 2 , Siren Rettedal 3 4 , Vilde Kolstad 4 and Kjersti Engan 1 5 · 2025

Read Two-Stream Thermal Imaging Fusion for Enhanced Time of Birth Detection in Neonatal Care by Jorge García-Torres 1 , Øyvind Meinich-Bache 1 2 , Sara Brunner 2 , Siren Rettedal 3 4 , Vilde Kolstad 4 and Kjersti Engan 1 5 (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-03244.txt.

Introduction

Jorge García-Torres and co-authors build a system that finds the exact time of birth from thermal video in delivery rooms, to aid newborn care. About one in ten newborns needs help to start breathing, so a precise birth time guides fast care. Hand recording of that time is often wrong, which is the gap the system fills.

Two streams are fused: still image analysis and video motion analysis of the thermal recordings. Together they catch richer space and time signs of the birth moment than either stream alone. On short clips it reached 95.7 percent precision and 84.8 percent recall. A score joining step found the birth time in every test case, with a median error of two seconds.

Catalog record

CREATORJorge García-Torres 1 , Øyvind Meinich-Bache 1 2 , Sara Brunner 2 , Siren Rettedal 3 4 , Vilde Kolstad 4 and Kjersti Engan 1 5
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE21 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2503-03244

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Two-Stream Thermal Imaging Fusion for Enhanced Time of Birth Detection in Neonatal Care Jorge García-Torres 1 , Øyvind Meinich-Bache 1 2 , Sara Brunner 2 , Siren Rettedal 3 4 , Vilde Kolstad 4 and Kjersti Engan 1 5 Abstract Around 10% of newborns require some help to initiate breathing, and 5% need ventilation assistance. Accurate Time of Birth (ToB) documentation is essential for optimizing neonatal care, as timely interventions are vital for proper resuscitation. However, current clinical methods for recording ToB often rely on manual processes, which can be prone to inaccuracies. In this study, we present a novel two-stream fusion system that combines the power of image and video analysis to accurately detect the ToB from thermal recordings in the delivery room and operating theater. By integrating static and dynamic streams, our approach captures richer birth-related spatiotemporal features, leading to more robust and precise ToB estimation. We demonstrate that this synergy between data modalities enhances performance over single-stream approaches. Our system achieves 95.7% precision and 84.8% recall in detecting birth within short video clips. Additionally, with the help of a score aggregation module, it successfully identifies ToB in 100% of test cases, with a median absolute error of 2 seconds and an absolute mean deviation of 4.5 seconds compared to manual annotations. I Introduction Fig. 1: Overview of our proposed two-stream system for Time of Birth detection. At each second, a sliding window containing the previous 25 frames (3 seconds) is utilized as input x(t) to our model. The video clip is processed independently in each stream by pretrained backbones, extracting static and temporal features that are concatenated to produce richer probability scores p_{fusion} . These scores pass through a score aggregation module where they are combined with image-specific scores p_{vnb} and processed by an auxiliary model using a moving window w to refine ToB predictions. Globally, 10% of newborns require resuscitative interventions to start breathing at birth, with birth asphyxia causing nearly 900,000 deaths annually [ 1 , 2 ] . Neonatal resuscitation guidelines [ 3 , 4 ] emphasize initiating interventions within the “golden minute” after birth, making accurate Time of Birth (ToB) documentation essential for quality improvement, documentation, debriefing, and potential real-time decision support [ 5 ] . However, ToB–when the newborn is fully visible outside the mother’s perineum–is typically recorded manually with minute precision, limiting the reliability of Newborn Resuscitation Algorithm Activities (NRAA) timelines for clinical use and evidence-based research. AI has shown promise in automating these timelines [ 6 , 7 ] , but current approaches still depend on manually recorded ToB, restricting their potential. The NewbornTime project [ 8 ] seeks to bridge this gap by automating NewbornTimeline, a system designed to generate precise NRAA timelines, including ToB and resuscitation activities. This enables large-scale analysis of newborn resuscitation videos, producing NRAA timelines that can refine medical guidelines and improve neonatal care. A key innovation is the use of thermal imaging for ToB detection, facilitating GDPR-compliant privacy protection while leveraging newborns’ higher skin temperature for birth identification. Earlier work in this project introduced the first AI-driven ToB detector [ 9 ] , which relied solely on static images. By continuously analyzing individual thermal frames, this method effectively detected the newborn’s presence but sometimes lacked precision in identifying the exact moment of birth. To improve accuracy, we later proposed a video-based system [ 10 ] that extracted and classified short video clips from birth episodes. This system captured dynamic movements and thermal changes throughout the video sequence, achieving more accurate ToB estimations. However, this improvement came at the cost of an increased risk of missing detections. In this study, building on our previous approaches [ 9 , 10 ] , we integrate static spatial analysis and motion pattern modeling into a unified system, combining the strengths of both methods. Several studies [ 11 , 12 , 13 , 14 , 15 ] have explored the fusion of image and video data for human activity recognition, demonstrating that it can enhance AI systems. Here, we introduce a two-stream system for ToB detection that effectively processes both data modalities in a complementary manner. By employing a fusion strategy that combines both feature-level and decision-level fusion, the system leverages information about the newborn’s visibility (image stream) and birth moment (video stream) in parallel to make more informed decisions. Our method improves accuracy in ToB detection while reducing the risk of missed detections, leading to more stable and reliable outcomes. To the best of our knowledge, no other work has addressed automatic ToB detection. II Data Material The dataset comprises 611 thermal birth videos recorded semi-automatically [ 16 ] at Stavanger University Hospital (SUS), Norway. A passive thermal module by Mobotix [ 17 ] was installed in eight delivery rooms and one operating theater to capture the recordings. ToB was initially logged by midwives or nurse assistants using the Liveborn Observation App [ 18 ] , a tool designed for documenting post-birth events in research settings, enabling more precise ToB recording than standard clinical practice [ 19 ] . After registering the ToB in Liveborn, a 30-minute thermal video (resolution: 252 \times 336, frame rate: 8.33 fps) is saved, covering 15 minutes before and after birth, while the remaining footage is discarded. Due to the numerous tasks involved in childbirth, the registration of ToB in Liveborn was sometimes delayed. To ensure accurate ToB records, manual annotation with second precision was done by reviewing the thermal videos. 258 videos also included an additional label indicating whether the newborn was clearly visible and identifiable in the video frames (Visible Newborn, VNB). Video annotation was carried out by medical professionals using ELAN 5.8 (The Language Archive, Nijmegen, The Netherlands) [ 20 ] . III Methodology We propose a unified system that leverages complementary visual cues from thermal imaging for ToB detection. It involves three learning steps. We first train EfficientNet and MoViNet on birth episodes using their respective data modalities. Next, we develop a two-stream model that integrates image and video patterns through feature-level fusion. Finally, we implement a score aggregation module that performs decision-level fusion to refine the final ToB detection. This approach can be seen as a hybrid fusion where each step is learned independently. An overview of our method is shown in Figure 1 . III-A Preprocessing Previous work [ 21 ] identified key challenges in using thermal sensors for ToB detection, highlighting the limitations of relying on absolute measurements. To address this, we proposed a Gaussian Mixture Model (GMM) normalization method [ 9 ] that models the temperature distribution around human skin temperature, enhancing consistency and preserving relative temperature variations. Let \mathcal{V}\in\mathbb{R}^{N\times H\times W} represent a single-channel thermal video consisting of N frames of spatial resolution of H\times W . We define I(n) as the thermal frame at index n=0,1,\dots,N-1 . A video clip V(n) is then constructed using the preceding F frames: V(n)=\{I(n),I(n-1),...,I(n-F+1)\} (1) To normalize \mathcal{V} , a set of intensity values \mathbf{v} are extracted and fitted to the GMM. The highest Gaussian mean \hat{\mu}_{\mathbf{v}} , selected by empirical constraints, is then used to define a temperature range of interest. Our proposed normalization function \phi_{\hat{\mu}_{\mathbf{v}}} clips the temperature values within this video-specific range and rescales

Read the complete text