STDnet-ST: Spatio-temporal ConvNet for small object detection. (August 2021)

• Record Type: Journal Article
• Authors: Bosquet, Brais; Mucientes, Manuel; Brea, Víctor M.
• Is Part Of: Pattern recognition. Volume 116(2021)
• Publication Date: 2021-08
• Subjects: Small object detection -- Spatio-temporal convolutional network -- Object linking; Pattern perception -- Periodicals; Perception des structures -- Périodiques; Patroonherkenning; 006.4
• DOI: 10.1016/j.patcog.2021.107929
• Languages: English
• ISSNs: 0031-3203

Abstract

Highlights: STDnet-ST is a novel spatio-temporal ConvNet for small object detection. STDnet-ST exploits the correlation of promising regions between frames. An efficient tubelet linking is performed to link small objects across video frames. A novel tubelet suppression algorithm is proposed to avoid unprofitable tubelets. STDnet-ST outperforms its state-of-the-art counterparts for small target detection. Abstract: Object detection through convolutional neural networks is reaching unprecedented levels of precision. However, a detailed analysis of the results shows that the accuracy in the detection of small objects is still far from being satisfactory. A recent trend that will likely improve the overall object detection success is to use the spatial information operating alongside temporal video information. This paper introduces STDnet-ST, an end-to-end spatio-temporal convolutional neural network for small object detection in video. We define small as those objects under 16 × 16 px, where the features become less distinctive. STDnet-ST is an architecture that detects small objects over time and correlates pairs of the top-ranked regions with the highest likelihood of containing those small objects. This permits to link the small objects across the time as tubelets. Furthermore, we propose a procedure to dismiss unprofitable object links in order to provide high quality tubelets, increasing the accuracy. STDnet-ST is evaluated on the publicly accessible USC-GRAD-STDdb, UAVDT and VisDrone2019-VID video datasets, where it achieves state-of-the-art results for small objects.