A novel one-stage framework for visual pulse rate estimation using deep neural networks. (April 2021)

• Record Type: Journal Article
• Authors: Huang, Bin; Lin, Chun-Liang; Chen, Weihai; Juang, Chia-Feng; Wu, Xingming
• Is Part Of: Biomedical signal processing and control. Volume 66(2021)
• Publication Date: 2021-04
• Subjects: Visual pulse rate estimation -- Remote PPG -- Pulse rate monitoring -- Deep learning -- 3D convolutional neural networks -- LSTM (long short-term memory); Signal processing -- Periodicals; Biomedical engineering -- Periodicals; Signal Processing, Computer-Assisted -- Periodicals; Image Processing, Computer-Assisted -- Periodicals; Biomedical Engineering -- Periodicals; 610.28
• DOI: 10.1016/j.bspc.2020.102387
• Languages: English
• ISSNs: 1746-8094

Abstract

Highlights: PRnet can extract PR with a 2s video, and adapt for a sharp variation PR. PRnet is a complete neural networks pipeline with joint optimization framework. PRnet is a spatio-temporal frramwork that can extract the latent temporal information. PRnet achieves competitive performance than state-of-the-art methods. Abstract: Estimation of the visual pulse rate (also called heart rate) refers to extraction of the pulse rate from a facial video. With the studies on extracting photoplethysmography (PPG) signals from a facial video, the non-contacted measurement method has aroused great interest among researchers over the past few years. In this study, a novel one-stage spatio-temporal framework, namely PRnet, is proposed to estimate the pulse rate from a stationary facial video. First, visual pulse rate estimation is defined as a regression task based on deep neural networks, in which a video is mapped to a pulse rate value. Then, 3D convolutional neural networks (Conv3D) and Long short-term memory (LSTM) modules are used to extract spatial and latent temporal information that is hidden in a video. Subsequently, one fully connected layer is applied in the last layer of PRnet to estimate the pulse rate directly. Based on the exquisite framework design, our proposed method realizes competitive performance, especially in terms of processing latency, since it does not rely on power spectral density (PSD) and traditional Fast Fourier Transform (FFT) algorithms. Using our method, only 60 frames of video (2 s) are required for the robust prediction of the pulse rate, whereas 6–30 s of video are typically required for other methods. Finally, a novel visual pulse rate estimation database, which includes pulse rate range at various times of day, is collected to evaluate the proposed framework. The results of extensive experiments demonstrate that PRnet performs competitively while compared with state-of-the-art methods.