Human-related anomalous event detection via memory-augmented Wasserstein generative adversarial network with gradient penalty. (June 2023)

• Record Type: Journal Article
• Authors: Li, Nanjun; Chang, Faliang; Liu, Chunsheng
• Is Part Of: Pattern recognition. Volume 138(2023)
• Publication Date: 2023-06
• Subjects: Human-related anomalous event detection -- Video surveillance -- Human skeleton trajectories -- Wasserstein generative adversarial network with gradient penalty -- Memory module; Pattern perception -- Periodicals; Perception des structures -- Périodiques; Patroonherkenning; 006.4
• DOI: 10.1016/j.patcog.2023.109398
• Languages: English
• ISSNs: 0031-3203

Abstract

Highlights: A MemWGAN-GP network is proposed to predict future human skeleton trajectories from a given past and reconstruct the given past simultaneously for detecting abnormal human behaviors. An adversarial loss based on the Wasserstein distance between the generator and critic of the MemWGAN-GP is utilized to improve the reconstruction and prediction ability. The 2D convolutional and deconvolutional layers are employed in the generator to model the temporal patterns of input trajectory feature sequence. Abstract: Timely detection of human-related anomaly in surveillance videos is a challenging task. Generally, the irregular human motion and action patterns can be regarded as abnormal human-related events. In this paper, we utilize the skeleton trajectories to learn the regularities of human motion and action in videos for anomaly detection. The skeleton trajectories are decomposed into global and local feature sequences, which are utilized to provide human motion and action information, respectively. Then, the global and local sequences are modeled as two separate sub-processes with our proposed Memory-augmented Wasserstein Generative Adversarial Network with Gradient Penalty (MemWGAN-GP). In each sub-process, the pre-trained MemWGAN-GP is employed to predict future feature sequences from corresponding input past sequences and reconstruct the input sequences simultaneously. The predicted and reconstructed feature sequences are compared with their groundtruth to identify anomalous sequences. The MemWGAN-GP integrates the autoencoder with a WGAN model to boost the reconstruction and prediction ability of the autoencoder. Besides, a memory module is employed in MemWGAN-GP to overcome high capacity of the autoencoder for anomalies reconstruction and prediction. Experimental results on four challenging datasets demonstrate advantages of the proposed method over other state-of-the-art algorithms.