Semantic annotation for complex video street views based on 2D–3D multi-feature fusion and aggregated boosting decision forests. (February 2017)

• Record Type: Journal Article
• Authors: Wang, Xun; Yan, Guoli; Wang, Huiyan; Fu, Jianhai; Hua, Jing; Wang, Jingqi; Yang, Yutao; Zhang, Guofeng; Bao, Hujun
• Is Part Of: Pattern recognition. Volume 62(2017:Feb.)
• Page Start: 189
• Page End: 201
• Publication Date: 2017-02
• Subjects: Semantic annotation -- Superpixel segmentation -- 2D–3D feature fusion -- ABDF model; Pattern perception -- Periodicals; Perception des structures -- Périodiques; Patroonherkenning; 006.4
• DOI: 10.1016/j.patcog.2016.08.030
• Languages: English
• ISSNs: 0031-3203

Abstract

Abstract: Accurate and efficient semantic annotation is an important but difficult step in large-scale video interpretation. This paper presents a novel framework based on 2D–3D multi-feature fusion and aggregated boosting decision forest (ABDF) for semantic annotation of video street views. We first integrate the 3D and 2D features to define the appearance model for characterizing the different types of superpixels and the similarities between two adjacent superpixel blocks. We then propose the ABDF algorithm to build the weak classifier by using a modified integrated splitting strategy for decision trees. And a Markov random field is then adopted to perform global superpixel block optimization to correct the minor errors and make the boundary for semantic annotation smoother. Finally, a boosting strategy is used to aggregate the different weak decision trees into one final strong classification decision tree. The superpixel block instead of the pixel is used as the basic processing unit, thus only a small amount of features are required to build an accurate and efficient model. The experimental results demonstrate the advantages of the proposed method in terms of classification accuracy and computation efficiency over those of existing semantic segmentation methods. The proposed framework can be used in real-time video processing applications. Abstract : Highlights: 2D and 3D superpixel features for object representation. A modified aggregated boosting decision forest for fast and accurate classification. Only a small amount of samples for building classification model. An effective tool for accurate and efficient semantic annotation of complex street views. Better performance in accuracy, robustness and computation efficiency.