RPPG-FuseNet: Non-contact heart rate estimation from facial video via RGB/MSR signal fusion. (September 2022)

• Record Type: Journal Article
• Authors: Jaiswal, Kokila Bharti; Meenpal, T.
• Is Part Of: Biomedical signal processing and control. Volume 78(2022)
• Publication Date: 2022-09
• Subjects: Non contact heart rate measurement -- Convolutional neural network -- Illumination variation -- rPPG -- Spatiotemporal map; Signal processing -- Periodicals; Biomedical engineering -- Periodicals; Signal Processing, Computer-Assisted -- Periodicals; Image Processing, Computer-Assisted -- Periodicals; Biomedical Engineering -- Periodicals; 610.28
• DOI: 10.1016/j.bspc.2022.104002
• Languages: English
• ISSNs: 1746-8094

Abstract

Abstract: Remote photoplethysmography (rPPG) is an effective way of measuring Heart Rate (HR) from facial video of a person. Several methods have been proposed but measurement in realistic situation i.e, uncontrolled motion and illumination, is still a challenging task. It is observed that the performance of state-of-the-art methods often gets degraded in weekly illuminated environment. In this paper, a simple fusion based approach is proposed to reduce the performance degradation due to non-uniform illumination. An algorithm is proposed to fuse two complimentary color spaces i.e, Red, Green and Blue (RGB) and Multi Scale Retinex (MSR). The methodology includes salient spatiotemporal map generation from fusion of RGB and MSR. CNN is employed to map spatiotemporal features to corresponding HR value. To prove the efficacy of the proposed method, comparison is done with the state-of-the art methods with and without deep learning approach. Achieved experimental results proves that the proposed method yields comparable result for both inter and intra database testing on publicly available databases. Highlights: A novel HR estimation method, using RGB and MSR images for varying illumination. Spatiotemporal maps are formed to improve the HR detection accuracy. CNN is used to estimate HR from spatiotemporal maps.