VFMVAC: View-filtering-based multi-view aggregating convolution for 3D shape recognition and retrieval. (September 2022)

• Record Type: Journal Article
• Authors: Liu, Zehua; Zhang, Yuhe; Gao, Jian; Wang, Shurui
• Is Part Of: Pattern recognition. Volume 129(2022)
• Publication Date: 2022-09
• Subjects: Multi-view -- Channel shuffle -- Convolution -- Recognition -- Retrieval; Pattern perception -- Periodicals; Perception des structures -- Périodiques; Patroonherkenning; 006.4
• DOI: 10.1016/j.patcog.2022.108774
• Languages: English
• ISSNs: 0031-3203

Abstract

Highlights: We propose a high-precision 3D shape multi-view recognition framework which can highly promote the performance of the 3D shape classification and retrieval. A voting-based view filtering algorithm is proposed; this algorithm can select the most representative views among the existing views to represent 3D shapes, thereby significantly improving memory usage efficiency and reducing the computational cost. A novel multi-view aggregating module is designed; in particular, the k-view features are shuffled using a cross-view channel shuffle module that considers the combination of features across views, thereby allowing for their sufficient fusion; furthermore, this module fuses the multi-view features via an aggregating convolution and considers all features of each view, thereby avoiding information loss induced by the traditional pooling methods. The proposed framework achieves state-of-the-art recognition and retrieval performance on benchmark datasets. Abstract: Multi-view based 3D shape recognition methods have achieved state-of-the-art performance in 3D shape recognition and retrieval. The main focus of multi-view based approaches is determining how to fuse multi-view features into a compact, descriptive, and robust 3D shape descriptor that can then be utilized for 3D shape recognition and retrieval. This paper proposes a novel multi-view aggregating framework, view-filtering-based multi-view aggregating convolution (VFMVAC) to learn global shape descriptors for 3D shape recognition. The proposed VFMVAC applies a voting-based view filtering strategy to select representative views, also introduces a novel multi-view aggregating module to integrate multi-view features; this substantially improves the descriptiveness of the descriptors, and therefore improves the performance of 3D shape recognition and retrieval. Specifically, all views are fed into a voting-based view filtering module to select the top-k representative views. Subsequently, the features of the top-k views are fed into the multi-view aggregating module, which first conducts cross-view channel shuffle for achieving cross-view information flowing, and the resulted reshaped features are then fed into the aggregating convolution module for feature fusion. Experiments on benchmark datasets demonstrate that the proposed VFMVAC is effective and outperforms several recent techniques with respect to the classification and retrieval performance, robustness and efficiency.