X-iPPGNet: A novel one stage deep learning architecture based on depthwise separable convolutions for video-based pulse rate estimation. (March 2023)

• Record Type: Journal Article
• Authors: Ouzar, Yassine; Djeldjli, Djamaleddine; Bousefsaf, Frédéric; Maaoui, Choubeila
• Is Part Of: Computers in biology and medicine. Volume 154(2023)
• Publication Date: 2023-03
• Subjects: Pulse rate estimation -- Convolutional neural networks -- End-to-end learning -- Imaging photoplethysmography -- Xception network; Medicine -- Data processing -- Periodicals; Biology -- Data processing -- Periodicals; 610.285
• DOI: 10.1016/j.compbiomed.2023.106592
• Languages: English
• ISSNs: 0010-4825

Abstract

Abstract: Pulse rate (PR) is one of the most important markers for assessing a person's health. With the increasing demand for long-term health monitoring, much attention is being paid to contactless PR estimation using imaging photoplethysmography (iPPG). This non-invasive technique is based on the analysis of subtle changes in skin color. Despite efforts to improve iPPG, the existing algorithms are vulnerable to less-constrained scenarios (i.e., head movements, facial expressions, and environmental conditions). In this article, we propose a novel end-to-end spatio-temporal network, namely X-iPPGNet, for instantaneous PR estimation directly from facial video recordings. Unlike most existing systems, our model learns the iPPG concept from scratch without incorporating any prior knowledge or going through the extraction of blood volume pulse signals. Inspired by the Xception network architecture, color channel decoupling is used to learn additional photoplethysmographic information and to effectively reduce the computational cost and memory requirements. Moreover, X-iPPGNet predicts the pulse rate from a short time window (2 s), which has advantages with high and sharply fluctuating pulse rates. The experimental results revealed high performance under all conditions including head motions, facial expressions, and skin tone. Our approach significantly outperforms all current state-of-the-art methods on three benchmark datasets: MMSE-HR ( M A E = 4.10 ; R M S E = 5.32 ; r = 0.85), UBFC-rPPG ( M A E = 4.99 ; R M S E = 6.26 ; r = 0.67), MAHNOB-HCI ( M A E = 3.17 ; R M S E = 3.93 ; r = 0.88). Graphical abstract: Highlights: X-iPPGNet can merge iPPG signal extraction and pulse rate prediction in one step. Depthwise separable convolution learns relevant features from each channel separately. X-iPPGNet is more suitable for real-time measurements and sharp fluctuation in PR. The proposed method achieves good performance in less-constrained scenarios. X-iPPGNet outperforms existing methods on three popular benchmark datasets.