Classification of human's activities from gesture recognition in live videos using deep learning. (6th January 2022)

• Record Type: Journal Article
• Authors: Khan, Amjad Rehman; Saba, Tanzila; Khan, Muhammad Zeeshan; Fati, Suliman Mohamed; Khan, Muhammad Usman Ghani
• Is Part Of: Concurrency and computation. Volume 34:Number 10(2022)
• Page Start: n/a
• Page End: n/a
• Publication Date: 2022-01-06
• Subjects: disaster risk reduction -- effective institutions -- ensemble learning -- object detection -- RCNN -- ROI -- smart invigilation | technological development; Parallel processing (Electronic computers) -- Periodicals; Parallel computers -- Periodicals; 004.35
• DOI: 10.1002/cpe.6825
• Languages: English
• ISSNs: 1532-0626

Abstract

Abstract: The automated behavior analysis is a significant problem in the examination. In the presence of invigilators and examiners, examinees use various unfair means to cheat the staff. Examinees may use different gestures like moving their head, eyes and using different suspicious objects (mobile phone, calculator) and so forth. This research is intended to automatically identify and distinguish examinees in live videos who cheat by their activities in examination hall during exams. Hence, we took the privilege of ensemble learning using deep learning‐based algorithms. The suspicious head movements and prohibited objects have been monitored using fine‐tuned Faster RCNN algorithm. At the same time, interactive use‐age of prohibited objects find out using Open‐Pose architecture. For this purpose, intersection over union (IOU) has been calculated between the region of interest (ROI) of detected objects and pose points of hands and face. To get the identity of a specific examinee, we stored the features of facial ROI after the last convolution layer and matched them at the time of testing for statistical analysis. We achieved state‐of‐the‐art results using standard evaluating measures. Moreover, for experiments, dataset of automatic examinee invigilation is available currently. So, this research also contributes toward the generation and annotation of the dataset.