An intelligent system for complex violence pattern analysis and detection. Issue 12 (5th July 2021)

• Record Type: Journal Article
• Authors: Ullah, Fath U Min; Obaidat, Mohammad S.; Muhammad, Khan; Ullah, Amin; Baik, Sung Wook; Cuzzolin, Fabio; Rodrigues, Joel J. P. C.; de Albuquerque, Victor Hugo C
• Is Part Of: International journal of intelligent systems. Volume 37:Issue 12(2022)
• Page Start: 10400
• Page End: 10422
• Publication Date: 2021-07-05
• Subjects: artificial intelligence -- complex pattern analysis -- convolutional neural network -- intelligent system -- LSTM -- violence detection; Artificial intelligence -- Periodicals; Expert systems (Computer science) -- Periodicals; Intelligence artificielle -- Périodiques; Systèmes experts (Informatique) -- Périodiques; 006.3
• DOI: 10.1002/int.22537
• Languages: English
• ISSNs: 0884-8173

Abstract

Abstract: Video surveillance has shown encouraging outcomes to monitor human activities and prevent crimes in real time. To this extent, violence detection (VD) has received substantial attention from the research community due to its vast applications, such as ensuring security over public areas and industrial settings through smart machine intelligence. However, because of changing illumination, complex background and low resolution, the analysis of violence patterns remains challenging in the industrial video surveillance domain. In this paper, we propose a computationally intelligent VD approach to precisely detect violent scenes through deep analysis of surveillance video sequential patterns. First, the video stream acquired through the vision sensor is processed by a lightweight convolutional neural network (CNN) for the segmentation of important shots. Next, temporal optical flow features are extracted from the informative shots via a residential optical flow CNN. These are concatenated with appearance‐invariant features extracted from a Darknet CNN model. Finally, a multilayer long short‐term memory network is plugged to generate the final feature map for learning the violence patterns in a sequence of frames. In addition, we contribute to the existing surveillance VD data set by considering its indoor and outdoor scenarios separately for the proposed method's evaluation, achieving a 2% increase in accuracy over surveillance fight data set. Experiments also show encouraging results over the state of the art on other challenging benchmark data sets.