RPPG-FuseNet: Non-contact heart rate estimation from facial video via RGB/MSR signal fusion. (September 2022) - Jaiswal, Kokila Bharti; Meenpal, T.
Abstract: Remote photoplethysmography (rPPG) is an effective way of measuring Heart Rate (HR) from facial video of a person. Several methods have been proposed but measurement in realistic situation i.e, uncontrolled motion and illumination, is still a challenging task. It is observed that the performance of state-of-the-art methods often gets degraded in weekly illuminated environment. In this paper, a simple fusion based approach is proposed to reduce the performance degradation due to non-uniform illumination. An algorithm is proposed to fuse two complimentary color spaces i.e, Red, Green and Blue (RGB) and Multi Scale Retinex (MSR). The methodology includes salient spatiotemporal map generation from fusion of RGB and MSR. CNN is employed to map spatiotemporal features to corresponding HR value. To prove the efficacy of the proposed method, comparison is done with the state-of-the art methods with and without deep learning approach. Achieved experimental results proves that the proposed method yields comparable result for both inter and intra database testing on publicly available databases. Highlights: A novel HR estimation method, using RGB and MSR images for varying illumination. Spatiotemporal maps are formed to improve the HR detection accuracy. CNN is used to estimate HR from spatiotemporal maps.