A vision-based hand hygiene monitoring approach using self-attention convolutional neural network. (July 2022)

• Record Type: Journal Article
• Authors: Xie, Tianming; Tian, Jing; Ma, Lihong
• Is Part Of: Biomedical signal processing and control. Volume 76(2022)
• Publication Date: 2022-07
• Subjects: Hand hygiene monitoring -- Handwashing recognition -- Convolutional neural network -- Self-attention mechanism -- Feature aggregation; Signal processing -- Periodicals; Biomedical engineering -- Periodicals; Signal Processing, Computer-Assisted -- Periodicals; Image Processing, Computer-Assisted -- Periodicals; Biomedical Engineering -- Periodicals; 610.28
• DOI: 10.1016/j.bspc.2022.103651
• Languages: English
• ISSNs: 1746-8094

Abstract

Highlights: We build a new benchmark handwashing action dataset. We apply self-attention convolutional neural network for handwashing recognition. The proposed approach is able to achieve more accurate recognition results. Abstract: Automatic hand hygiene monitoring is an effective and necessary practice in healthcare. The main objective is to automate the quality control of the hand hygiene process by recognizing handwashing actions. This paper proposes a new computer vision-based hand hygiene monitoring approach, with the aim to recognize handwashing actions from the input videos. The proposed approach consists of the following three modules. First, in the feature extraction module, a backbone convolutional neural network (CNN) model (i.e., a ResNet model) is used to extract features from each frame of the input video. Second, in the feature aggregation module, the self-attention mechanism is introduced to form six masked self-attention blocks that are stacked to aggregate features from multiple frames. Third, in the feature classification module, the self-attentive aggregated feature representation is used for handwashing action recognition. The proposed approach outperforms a few state-of-the-art handwashing action recognition approaches. The proposed approach is able to perform effective and accurate handwashing action recognition for automatic hand hygiene monitoring, as verified by three benchmark datasets.