Continuous action segmentation and recognition using hybrid convolutional neural network‐hidden Markov model model. Issue 6 (19th February 2016)

• Record Type: Journal Article
• Authors: Lei, Jun; Li, Guohui; Zhang, Jun; Guo, Qiang; Tu, Dan
• Is Part Of: IET computer vision. Volume 10:Issue 6(2016)
• Page Start: 537
• Page End: 544
• Publication Date: 2016-02-19
• Subjects: video signal processing -- image segmentation -- image recognition -- neural nets -- hidden Markov models -- Gaussian processes; continuous action segmentation -- continuous action recognition -- hybrid convolutional neural network-hidden Markov model model -- isolated action recognition -- convolutional neural network -- HMM -- statistical dependences -- CNN-HMM -- Gaussian mixture model -- Viterbi algorithm; Computer vision -- Periodicals; Pattern recognition systems -- Periodicals; 006.37
• DOI: 10.1049/iet-cvi.2015.0408
• Languages: English
• ISSNs: 1751-9632

Abstract

Abstract : Continuous action recognition in video is more complicated compared with traditional isolated action recognition. Besides the high variability of postures and appearances of each action, the complex temporal dynamics of continuous action makes this problem challenging. In this study, the authors propose a hierarchical framework combining convolutional neural network (CNN) and hidden Markov model (HMM), which recognises and segments continuous actions simultaneously. The authors utilise the CNN's powerful capacity of learning high level features directly from raw data, and use it to extract effective and robust action features. The HMM is used to model the statistical dependences over adjacent sub‐actions and infer the action sequences. In order to combine the advantages of these two models, the hybrid architecture of CNN‐HMM is built. The Gaussian mixture model is replaced by CNN to model the emission distribution of HMM. The CNN‐HMM model is trained using embedded Viterbi algorithm, and the data used to train CNN are labelled by forced alignment. The authors test their method on two public action dataset Weizmann and KTH. Experimental results show that the authors' method achieves improved recognition and segmentation accuracy compared with several other methods. The superior property of features learnt by CNN is also illustrated.