Player Identification in Hockey Broadcast Videos. (1st March 2021)

• Record Type: Journal Article
• Authors: Chan, Alvin; Levine, Martin D.; Javan, Mehrsan
• Is Part Of: Expert systems with applications. Volume 165(2021)
• Publication Date: 2021-03-01
• Subjects: Computer vision -- Recurrent models -- Convolutional neural networks -- Sports player identification -- Jersey numbers -- Broadcast hockey videos; Expert systems (Computer science) -- Periodicals; Systèmes experts (Informatique) -- Périodiques; Electronic journals; 006.33
• DOI: 10.1016/j.eswa.2020.113891
• Languages: English
• ISSNs: 0957-4174

Abstract

Abstract: We present a deep recurrent convolutional neural network (CNN) approach to solve the problem of hockey player identification in NHL broadcast videos. Player identification is a difficult computer vision problem mainly because of the players' similar appearance, occlusion, and blurry facial and physical features. However, we can observe players' jersey numbers over time by processing variable length image sequences of players (aka 'tracklets'). We propose an end-to-end trainable ResNet+LSTM network, with a residual network (ResNet) base and a long short-term memory (LSTM) layer, to discover spatio-temporal features of jersey numbers over time and learn long-term dependencies. Additionally, we employ a secondary 1-dimensional convolutional neural network classifier as a late score-level fusion method to classify the output of the ResNet+LSTM network. For this work, we created a new hockey player tracklet dataset that contains sequences of hockey player bounding boxes. This achieves an overall player identification accuracy score over 87% on the test split of our new dataset. Highlights: Recurrent convolutional neural network proposed for hockey player identification. Variable length image sequences of player bounding boxes (tracklets) are classified. Spatial–temporal information across video frames improves jersey number predictions. A secondary classifier added as a late score-level fusion method increases accuracy.