MVFNet: A multi-view fusion network for pain intensity assessment in unconstrained environment. (May 2021)

• Record Type: Journal Article
• Authors: Semwal, Ashish; Londhe, Narendra D.
• Is Part Of: Biomedical signal processing and control. Volume 67(2021)
• Publication Date: 2021-05
• Subjects: Pain assessment -- Patient monitoring -- Data fusion -- Deep neural network -- Pattern recognition; Signal processing -- Periodicals; Biomedical engineering -- Periodicals; Signal Processing, Computer-Assisted -- Periodicals; Image Processing, Computer-Assisted -- Periodicals; Biomedical Engineering -- Periodicals; 610.28
• DOI: 10.1016/j.bspc.2021.102537
• Languages: English
• ISSNs: 1746-8094

Abstract

Highlights: Facial expression based pain severity assessment in unconstrained environment is proposed. Cross dataset knowledge transfer based technique is used to reduce overfitting. Decision level fusion of three distinct features, i.e., RGB features, texture and complementary features are utilized. Introduced a new facial expression based pain dataset which is collected in unconstrained hospital environment. Proposed fusion model improves pain severity assessment up to 4%. Abstract: Pain is an indication of physical discomfort, and its monitoring is crucial for medical diagnosis and treatment of the patient. In the past few years, several techniques are proposed for pain assessment from face images. Although existing approaches provide satisfactory performance on constrained frontal faces, they might perform poorly in the natural unconstrained hospital environment due to low illumination conditions, large head-pose rotation, and occlusion which is common in an unconstrained environment. Therefore, a novel fusion approach to constitute discriminative features for pain severity assessment is proposed. In this work, decision level fusion of three distinct features, i.e., data-driven RGB features, entropy based texture features, and complementary features learned from both RGB and texture data are utilized to improve the generalization of the proposed pain assessment system. The experimental results demonstrate that the decision level fusion using these Multi-view features substantially outperforms the model trained with generic RGB data. Given this, the proposed system utilizes three CNNs, i.e., VGG-CNN based on cross dataset Transfer Learning (VGG-TL), Entropy Texture Network (ETNet), and Dual Stream CNN (DSCNN). Further, to alleviate the problem of overfitting various augmentation techniques are implemented. Furthermore, the proposed approach has been assessed extensively on self-generated datasets of 10 patients recorded in an unconstrained hospital environment. The experimental results demonstrate that the proposed model achieved 94.0 % of F1-score for pain severity assessment. In addition, to evaluate the generalization of the proposed method we also report competitive results in the UNBC-McMaster dataset.