Human activity recognition using tools of convolutional neural networks: A state of the art review, data sets, challenges, and future prospects. (October 2022)

• Record Type: Journal Article
• Authors: Islam, Md. Milon; Nooruddin, Sheikh; Karray, Fakhri; Muhammad, Ghulam
• Is Part Of: Computers in biology and medicine. Volume 149(2022)
• Publication Date: 2022-10
• Subjects: Human activity recognition -- Convolutional neural network -- Multimodal sensing devices -- Smartphone data -- Radar signal -- Vision systems; Medicine -- Data processing -- Periodicals; Biology -- Data processing -- Periodicals; 610.285
• DOI: 10.1016/j.compbiomed.2022.106060
• Languages: English
• ISSNs: 0010-4825

Abstract

Abstract: Human Activity Recognition (HAR) plays a significant role in the everyday life of people because of its ability to learn extensive high-level information about human activity from wearable or stationary devices. A substantial amount of research has been conducted on HAR and numerous approaches based on deep learning have been exploited by the research community to classify human activities. The main goal of this review is to summarize recent works based on a wide range of deep neural networks architecture, namely convolutional neural networks (CNNs) for human activity recognition. The reviewed systems are clustered into four categories depending on the use of input devices like multimodal sensing devices, smartphones, radar, and vision devices. This review describes the performances, strengths, weaknesses, and the used hyperparameters of CNN architectures for each reviewed system with an overview of available public data sources. In addition, a discussion of the current challenges to CNN-based HAR systems is presented. Finally, this review is concluded with some potential future directions that would be of great assistance for the researchers who would like to contribute to this field. We conclude that CNN-based approaches are suitable for effective and accurate human activity recognition system applications despite challenges including availability of data regarding composite or group activities, high computational resource requirements, data privacy concerns, and edge computing limitations. For widespread adaptation, future research should be focused on more efficient edge computing techniques, datasets incorporating contextual information with activities, more explainable methodologies, and more robust systems. Highlights: A comprehensive survey on the applications of convolutional neural networks in Human Activity Recognition. Review of HAR systems from four major modalities: multimodal sensing devices, smartphones, radars, and cameras. A comprehensive review of datasets for different modalities from a data availability perspective. In-depth analysis of the challenges, identification of possible future works, and exploration of the applications of HAR systems.