Weakly-supervised butterfly detection based on saliency map. (June 2023)

• Record Type: Journal Article
• Authors: Zhang, Ting; Waqas, Muhammad; Fang, Yu; Liu, Zhaoying; Halim, Zahid; Li, Yujian; Chen, Sheng
• Is Part Of: Pattern recognition. Volume 138(2023)
• Publication Date: 2023-06
• Subjects: Butterfly detection -- Saliency map -- Class activation map -- Weakly-supervised object detection; Pattern perception -- Periodicals; Perception des structures -- Périodiques; Patroonherkenning; 006.4
• DOI: 10.1016/j.patcog.2023.109313
• Languages: English
• ISSNs: 0031-3203

Abstract

Highlights: We extract the features of different scales using the VGG-16 without the fully-connected layers as the backbone network. The saliency maps of insects' images are extracted using the deep supervision network with shortcut connections. Used for the insects' target location. Class activation maps of insect images are derived via the adversarial complementary learning network for target recognition. Saliency and class activation maps are post-processed with conditional random fields The locations of the butterflies are acquired based on the saliency maps. Abstract: Given the actual needs for detecting multiple features of butterflies in natural ecosystems, this paper proposes a model of weakly-supervised butterfly detection based on a saliency map (WBD-SM) to enhance the accuracy of butterfly detection in the ecological environment as well as to overcome the difficulty of fine annotation. Our proposed model first extracts the features of different scales using the VGG16 without the fully connected layers as the backbone network. Next, the saliency maps of butterfly images are extracted using the deep supervision network with shortcut connections (DSS) used for the butterfly target location. The class activation maps of butterfly images are derived via the adversarial complementary learning (ACoL) network for butterfly target recognition. Then, the saliency and class activation maps are post-processed with conditional random fields, thereby obtaining the refined saliency maps of butterfly objects. Finally, the locations of the butterflies are acquired based on the saliency maps. Experimental results on the 20 categories of butterfly dataset collected in this paper indicate that the WBD-SM achieves a higher recognition accuracy than that of the VGG16 under different division ratios. At the same time, when the training set and test set are 8:2, our WBD-SM attains a 95.67% localization accuracy, which is 9.37% and 11.87% higher than the results of the DSS and ACoL, respectively. Compared with three state-of-the-art fully-supervised object detection networks, RefineDet, YOLOv3 and single-shot detection (SSD), the detection performance of our WBD-SM is better than RefineDet, and YOLOv3, and is almost the same as SSD.