MVRD-Bench: Multi-View Learning and Benchmarking for Dynamic Remote Photoplethysmography under Occlusion

Zuxian He, Xu Cheng, Zhaodong Sun, Haoyu Chen, Jingang Shi, Xiaobai Li, Guoying Zhao · 2026

Read MVRD-Bench: Multi-View Learning and Benchmarking for Dynamic Remote Photoplethysmography under Occlusion by Zuxian He, Xu Cheng, Zhaodong Sun, Haoyu Chen, Jingang Shi, Xiaobai Li, Guoying Zhao (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-22826.txt.

Introduction

He and co-authors build a benchmark and method for contact-free pulse reading from face video. Motion and occlusion hurt single-view systems. Their dataset, MVRD, holds synced face videos from three viewpoints. Scenarios cover still, speaking, and head movement cases. Their method fuses cues across views to keep skin coverage under motion. Parts handle motion artifacts, rhythm features, and view weighting. Tests on the new dataset beat prior methods. In the movement case, error stays very low and correlation near perfect.

Catalog record

CREATORZuxian He, Xu Cheng, Zhaodong Sun, Haoyu Chen, Jingang Shi, Xiaobai Li, Guoying Zhao
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE38874
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.22826

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

MVRD-Bench: Multi-View Learning and Benchmarking for Dynamic Remote Photoplethysmography under Occlusion Zuxian He, Xu Cheng, Zhaodong Sun, Haoyu Chen, Jingang Shi, Xiaobai Li, Guoying Zhao Abstract Remote photoplethysmography (rPPG) is a non-contact technique that estimates physiological signals by analyzing subtle skin color changes in facial videos. Existing rPPG methods often encounter performance degradation under facial motion and occlusion scenarios due to their reliance on static and single-view facial videos. Thus, this work focuses on tackling the motion-induced occlusion problem for rPPG measurement in unconstrained multi-view facial videos. Specifically, we introduce a M ulti- V iew r PPG D ataset (MVRD), a high-quality benchmark dataset featuring synchronized facial videos from three viewpoints under stationary, speaking, and head movement scenarios to better match real-world conditions. We also propose MVRD-rPPG, a unified multi-view rPPG learning framework that fuses complementary visual cues to maintain robust facial skin coverage, especially under motion conditions. Our method integrates an Adaptive Temporal Optical Compensation (ATOC) module for motion artifact suppression, a Rhythm-Visual Dual-Stream Network to disentangle rhythmic and appearance-related features, and a Multi-View Correlation-Aware Attention (MVCA) for adaptive view-wise signal aggregation. Furthermore, we introduce a Correlation Frequency Adversarial (CFA) learning strategy, which jointly enforces temporal accuracy, spectral consistency, and perceptual realism in the predicted signals. Extensive experiments and ablation studies on the MVRD dataset demonstrate the superiority of our approach. In the MVRD movement scenario, MVRD-rPPG achieves an MAE of 0.90 and a Pearson correlation coefficient (R) of 0.99. The source code and dataset will be made available. Introduction Remote photoplethysmography (rPPG) is of significance for healthcare ( MTTS-CAN ) , emotion analysis ( 29 ) . It aims to estimate physiological signals such as heart rate (HR), respiration frequency (RF), and heart rate variability (HRV) from facial videos in a non-contact manner. In recent years, rPPG has emerged as an innovative alternative for acquiring physiological signals, leveraging camera technology to detect subtle color fluctuations on human skin. Despite significant progress in rPPG ( 21 ; 22 ; 14 ; 4 ; 26 ; 27 ; 9 ; 18 ; 19 ; 28 ; 16 ; 6 ) , existing datasets and methods remain limited in dynamic scenarios, as illustrated in Fig. 1 . As shown in Fig. 1 (a), most existing rPPG methods operate on single-view facial videos. Under natural head movements, a single view can easily suffer from facial ROI occlusion and motion artifacts, which directly degrade the performance of rPPG estimation. Although motion compensation can reduce interference to some extent, it cannot solve the visibility loss caused by the single-view bottleneck. In Fig. 1 (b), Egorov et al. ( 6 ) introduced the first public multi-view MCD-rPPG dataset for analyzing view variations in rPPG. However, it is restricted to static scenarios and cannot capture realistic view changes caused by natural head movements. In addition, existing public multi-view datasets ( 24 ; 6 ) and the corresponding rPPG methods did not establish a dynamic multi-view fusion prototype for rPPG estimation, which limits the use of complementary facial visibility across views during motion. As a result, dynamic multi-view rPPG under motion-induced occlusion remains underexplored. To address these limitations, we focus on dynamic multi-view rPPG estimation, as shown in Fig. 1 (c). We construct a Multi-View rPPG Dataset (MVRD) with synchronized recordings from three viewpoints under stationary, speaking, and head movement scenarios, and further develop a unified multi-view rPPG learning framework that combines motion compensation with cross-view fusion for robust rPPG estimation under motion-induced occlusion. Figure 1: Motivation and challenges in the movement scenario. (a) Single-view rPPG remains vulnerable to motion-induced facial ROI occlusion even with motion compensation. (b) Existing multi-view data (e.g., MCD-rPPG) are captured independently in static settings and process views. (c) We construct MVRD dataset and develop a multi-view rPPG measurement framework for robust estimation in movement scenarios. Generally, the contributions of this work can be summarized as follows: • We construct a Multi-View Remote Photoplethysmography dataset (MVRD), which contains facial recordings from three viewpoints in stationary, speaking, and head movement scenarios to study the motion robustness under the multi-view setting. • We propose the first unified multi-view rPPG learning framework (MVRD-rPPG) utilizing visual cues of different views to maintain full facial skin coverage and improve motion robustness, encompassing three carefully designed components: Adaptive Temporal Optical Compensation module, Rhythm-Visual Dual-Stream Network, and Multi-View Correlation-Aware Attention. • We develop a Correlation Frequency Adversarial (CFA) learning strategy to capture more stable power spectral density distributions and improve rPPG signal measurement in fused prediction, which jointly enforces temporal accuracy, spectral consistency, and perceptual realism in the predicted signals. • Extensive experiments and ablation studies on MVRD and public datasets demonstrate the robustness and generalization ability of the proposed framework in both dynamic multi-view setting and common single-view setting. Related Work Video-Based Remote Physiological Measurement. Extracting rPPG signals from facial videos is challenging due to their subtle nature. Early signal processing methods relied on priors. Verkruysse ( 21 ) first demonstrated the feasibility of extracting physiological signals under ambient light. PCA and ICA ( 14 ) decompose temporal signals, while CHROM ( 5 ) and POS ( 22 ) project skin-color signals into subspaces to estimate HR using skin boundary and mask filters. With the development of deep learning, supervised-based rPPG models have been proposed to learn spatial-temporal features from raw frames and ground-truth signals. DeepPhys ( 4 ) introduces convolutional attention to disentangle appearance and motion. PhysNet ( 26 ) models temporal dynamics with 3D CNNs. MTTS-CAN ( 12 ) performs multi-task learning for pulse and attention, while TS-CAN ( 12 ; 11 ) improves temporal consistency via temporal shifts. However, these models are dependent on high-quality synchronized labels. Unsupervised and self-supervised rPPG methods have emerged to address this limitation. AutoHR ( 25 ) uses a spatial-temporal autoencoder for signal reconstruction. SiNC ( 16 ) introduces frequency-domain consistency constraints. Contrast-Phys ( 18 ; 19 ) employs a contrastive framework to pull together rPPG signals from the same video and separate those from different videos. Yue et al. ( 28 ) propose a frequency-inspired self-supervised learning strategy that learns rPPG signals from unlabeled facial videos via contrastive frequency constraints. These methods work well under stable frontal views but often degrade under occlusion or large head movement. Optical Flow for Motion Compensation. Head movements and facial expressions introduce non-rigid motion artifacts that degrade the rPPG signal quality. Optical flow has been utilized to estimate the pixel-level displacement between frames to reduce motion-related distortions. DeepPhys ( 4 ) uses frame difference maps to guide attention learning. Gong et al. ( 7 ) proposed a flow-guided attention mechanism to emphasize temporally consistent regions. These studies reveal that motion-aware representations improve robustness in the context of moderate movement. However, performance drops significantly under large head movements. Li et al. ( 10 ) utilized optical flow-based temporal alignment to suppress motion-induced noise in rPPG signals, enhancing robustness under head movement. Public Datasets for rPPG. Benchmark datasets have been crucial in advancing the development of rPPG models by providing standardized evaluation frameworks. Early datasets, such as UBFC-rPPG ( 3 ) and OBF ( 2 ) , were collected under conditions of small movement and consistent illumination, which limited their generalizability. The PURE ( 17 )

Read the complete text