Remote Photoplethysmography in Real-World and Extreme Lighting Scenarios
Hang Shao, Lei Luo, Jianjun Qian, Mengkai Yan, Shuo Chen, Jian Yang · 2025
Read Remote Photoplethysmography in Real-World and Extreme Lighting Scenarios by Hang Shao, Lei Luo, Jianjun Qian, Mengkai Yan, Shuo Chen, Jian Yang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-11465.txt.
Introduction
Hang Shao, Jian Yang, and co-authors build a video model that reads pulse from face film in wild light, in a 2025 paper. Tiny skin tone shifts with each heart beat can give pulse with no touch. Past tools fail when light is harsh or scenes are messy, as in real outdoor use.
Their model strips shared noise with help from scene backdrop cues. It then learns clean beat signs with self taught splits and beat based rules. Tests across data sets and scenes show strong scores. The model is light to run, and the team calls it the first fit for true outdoor face film.
Catalog record
| CREATOR | Hang Shao, Lei Luo, Jianjun Qian, Mengkai Yan, Shuo Chen, Jian Yang |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 42 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2503-11465
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Remote Photoplethysmography in Real-World and Extreme Lighting Scenarios Thanks: Key Lab of Intelligent Perception and Systems for High-Dimensional Information of Ministry of Education, School of Computer Science and Engineering. Hang Shao, Lei Luo, Jianjun Qian, Mengkai Yan, Shuo Chen, Jian Yang Abstract Physiological activities can be manifested by the sensitive changes in facial imaging. While they are barely observable to our eyes, computer vision manners can, and the derived remote photoplethysmography (rPPG) has shown considerable promise. However, existing studies mainly rely on spatial skin recognition and temporal rhythmic interactions, so they focus on identifying explicit features under ideal light conditions, but perform poorly in-the-wild with intricate obstacles and extreme illumination exposure. In this paper, we propose an end-to-end video transformer model for rPPG. It strives to eliminate complex and unknown external time-varying interferences, whether they are sufficient to occupy subtle biosignal amplitudes or exist as periodic perturbations that hinder network training. In the specific implementation, we utilize global interference sharing, subject background reference, and self-supervised disentanglement to eliminate interference, and further guide learning based on spatiotemporal filtering, reconstruction guidance, and frequency domain and biological prior constraints to achieve effective rPPG. To the best of our knowledge, this is the first robust rPPG model for real outdoor scenarios based on natural face videos, and is lightweight to deploy. Extensive experiments show the competitiveness and performance of our model in rPPG prediction across datasets and scenes. 1 Introduction Heart rate (HR), heart rate variability, respiratory rate, blood pressure, and blood oxygen saturation are significant vital parameters, and their remote sensing can provide important evidence for many tasks for clinical purposes and intelligent applications. Remote photoplethysmography (rPPG) aims to capture blood volume pulse (BVP) signals containing the above bio-indicators based on facial videos wang_tbme17 ; shao_tim24 . With the popularization of commercial cameras and mobile communication devices, rPPG technique will play an irreplaceable role in health self-examination, anti-spoofing, driver fatigue diagnosis, and DeepFake in the foreseeable future. Figure 1 : Existing learning-based rPPGs have shown promise under static illuminations (a), but they are not effective in real-world scenarios (b). Performances of different methods can be compared intuitively (under the MR-NIRP-DRV Nowara_tits22 dataset). The contraction and relaxation of cardiac alter the instantaneous flow and velocity of red blood cells in subcutaneous capillaries, thereby changing facial skin imaging. Although these contents are difficult to be directly perceived by naked eyes, camera can capture the absorption and scattering of incident light caused by variations in superficial living tissues haan_tbme13 ; shao_tnnls24 . On these bases, rPPG summarizes biosignal clues by comparing skin color between frames. Early related studies implemented rPPG based on manual features Tulyakov_cvpr16 ; Pilz_cvpr18 . Subsequently, deep learning-based methods showed stronger robustness in motion and heterogeneous videos yu_cvpr22 ; sun_pami24 . However, similar to the bottleneck of the traditional blind source separation methods which can demodulate as long as biosignal meets certain conditions, existing learning-based rPPG methods are also vulnerable in many cases, especially in the following aspects: 1) Insufficient ability to remove extreme light interference. Facial color changes caused by heart activity are relatively asthenic. When external light interference is sufficient to mask such changes, existing rPPG methods that mine clues from skin region-of-interest (ROI) can easily fail liu_nips20 ; yue_tim22 . 2) Weak ability to resist periodic interference. Cardiac signals have quasi-periodic characteristics, and many existing long-term BVP perception models are developed based on this. However, when the external interference also shows a certain time-varying and rhythmicity, it is more difficult to unravel lee_cvpr23 ; liu_arxiv24 . 3) Still need to employ additional expensive equipment in practice. For the unknown complex interference, some studies choose to introduce near-infrared (NIR) imaging as multimodal input, but this leads to the community return to the old routine of complicated devices and high costs chiu_tim23 ; gong_tim24 . 4) Difficulty in model deployment. Existing rPPG networks, especially those for the noise disentanglement, continue to accumulate heavy modules and complex pipelines, making them difficult to deploy on flexible hardware zhang_tcsvt24 ; liu_tip24 . In summary, in the real-world (such as driving scenarios) where the above challenges coexist, operating learning-based rPPG models rarely perform satisfactorily (as shown in Fig. 1 ). In this paper, we propose a novel rPPG framework based on vision transformer, which can be applied to a variety of application scenes with drastic illumination and complex in-the-wild interference, and can be better adapted to terminals due to its lightweight layout. Specifically, to prevent rPPG waveforms from being overwhelmed by external interference, our model exploits the common property of facial skin and subject background disturbances, takes the background interference as a reference, discouples it in a self-supervised manner, and eliminates its influence in the foreground. In this process, we increase the frame window length to make full utilize of BVP quasi-periodic features that are different from extreme lighting obstructions. Meanwhile, we use the biosignal frequency domain and biological prior constraints to prevent the influence of interference with rhythmic characteristics that are hidden in too long a temporal period, and use spatiotemporal filtering and reconstruction to eliminate sensitive noise and better guide network training. It is worth noting that all our learning support is based on facial optical reflectance model and feature mapping correlation calculation, without introducing additional imaging modalities and too many branch modules, so the deployment cost is lower. We conduct experiments on multiple mainstream datasets, including BUAA-MIHR (illuminance state variation xi_fg20 ), VIPL-HR (indoor real state niu_tip20 ), MR-NIRP-IND (motion Nowara_cvpr18 ), and MR-NIRP-DRV (outdoor driving moment Nowara_tits22 ). Sufficient verification results demonstrate that our approach shows significant advantages in actual open and challenging scenarios. In summary, the main contributions include: • We pioneer rPPG learning from facial videos in outdoor variable illumination, and propose a targeted paradigm. It relies merely on RGB imaging instead of various infrared spectra, aiming to extend rPPG to more realistic domains. • Our model is guided by global interference sharing, background perturbation reference, self-supervised disentanglement, and long-term frequency and physiological prior constraint strategies, which can provide a robust solution for resisting unknown complex obstructions. • Our model is lighter than existing noise disentanglement methods, can be deployed more flexibly, and is more suitable for in-the-wild platforms and real-world tasks. • We compare with the latest representative rPPG methods and demonstrate that our model achieves the state-of-the-art results in both extreme lighting and static scenes. 2 Related works 2.1 Learning-based rPPG in daily life Recently, researchers have developed many learning-based visual rPPG networks, which have continuously improved task-oriented performances in facial ROI attention liu_nips20 ; liu_wacv23 , spatiotemporal enhancement gupta_wacv23 ; zou_mamba24 , and long-term time perception yu_ijcv23 ; zou_24 . However, capabilities of these methods in open conditions are worrying. Currently, rPPG for actual scenes mainly solves the problems of subject motion robustness li_aaai23 ; Cantrill_cvpr24 and shallow denoising lu_cvpr21 ; du_cvpr23 ; savic_fg24 , and rarely involves extreme real-time changes in illumination. Even the most widely used VIPL-HR dataset niu_tip20 , which is generally considered to be the closest to real states, is not exception. Although some works have begun to focus on outdoor lighting, they all require the exploit of auxiliary infrared devices and have limited surface feature abilities Kurihara_tip21 ; xu_tim23 . Infrared imaging