Sensor-based and vision-based human activity recognition: A comprehensive survey. (December 2020)

• Record Type: Journal Article
• Authors: Minh Dang, L.; Min, Kyungbok; Wang, Hanxiang; Jalil Piran, Md.; Hee Lee, Cheol; Moon, Hyeonjoon
• Is Part Of: Pattern recognition. Volume 108(2020:Dec.)
• Publication Date: 2020-12
• Subjects: Human activity recognition -- Action recognition -- Sensors -- Vision -- Human-centric sensing -- Deep learning -- Context-awareness; Pattern perception -- Periodicals; Perception des structures -- Périodiques; Patroonherkenning; 006.4
• DOI: 10.1016/j.patcog.2020.107561
• Languages: English
• ISSNs: 0031-3203

Abstract

Highlights: A comprehensive review of vision-based and sensor-based human activity recognition. Summarize and discuss public datasets that are used in vision-based HAR and sensor-based HAR. Categorize and analyze standard data processing, and feature engineering processes used in HAR. Categorize and analyze machine learning techniques for HAR and focus on current deep learning research in HAR. Discuss challenges and show future directions for HAR. Abstract: Human activity recognition (HAR) technology that analyzes data acquired from various types of sensing devices, including vision sensors and embedded sensors, has motivated the development of various context-aware applications in emerging domains, e.g., the Internet of Things (IoT) and healthcare. Even though a considerable number of HAR surveys and review articles have been conducted previously, the major/overall HAR subject has been ignored, and these studies only focus on particular HAR topics. Therefore, a comprehensive review paper that covers major subjects in HAR is imperative. This survey analyzes the latest state-of-the-art research in HAR in recent years, introduces a classification of HAR methodologies, and shows advantages and weaknesses for methods in each category. Specifically, HAR methods are classified into two main groups, which are sensor-based HAR and vision-based HAR, based on the generated data type. After that, each group is divided into subgroups that perform different procedures, including the data collection, pre-processing methods, feature engineering, and the training process. Moreover, an extensive review regarding the utilization of deep learning in HAR is also conducted. Finally, this paper discusses various challenges in the current HAR topic and offers suggestions for future research.