Mask-guided explicit feature modulation for multispectral pedestrian detection. (October 2022)

• Record Type: Journal Article
• Authors: Shen, Jifeng; Liu, Yue; Chen, Yifei; Zuo, Xin; Li, Jun; Yang, Wankou
• Is Part Of: Computers & electrical engineering. Volume 103(2022)
• Publication Date: 2022-10
• Subjects: Multispectral pedestrian detection -- Feature modulation -- Mutual attention -- Score fusion; Computer engineering -- Periodicals; Electrical engineering -- Periodicals; Electrical engineering -- Data processing -- Periodicals; Ordinateurs -- Conception et construction -- Périodiques; Électrotechnique -- Périodiques; Électrotechnique -- Informatique -- Périodiques; Computer engineering; Electrical engineering; Electrical engineering -- Data processing; Periodicals; Electronic journals; 621.302854
• DOI: 10.1016/j.compeleceng.2022.108385
• Languages: English
• ISSNs: 0045-7906

Abstract

Abstract: Multi-task learning of object detection and box-level segmentation is commonly formulated as an implicit feature modulation method, which suffers lacking task interaction. In this paper, a novel explicit feature modulation solution with two different mask infusion methods is proposed. The first modulation method is semantic feature enhancement of backbone, which is achieved by a novel mask-guided mutual attention module (MMA). The proposed MMA module can explicitly guide the feature maps towards a more semantic informative direction for focalizing centrality of pedestrian, which can significantly improve the performance. The second modulation method is confidence score enhancement of detection head, which is benefited from our proposed mask-guided score fusion module (MSF). The proposed MSF module collects information from the classification, IOU, centerness feature map and the learned mask, which can discriminate false and true positives more effectively. It is qualitatively validated that the modulated feature maps in both backbone and detection head become more semantically meaningful and robust to scale and occlusion. Our method achieves a considerable gain over the state-of-the-arts on the KAIST, CVC14 and FLIR datasets. Besides, it runs at 22 FPS in default setting, making it favorable in many practical scenarios. Graphical abstract: Highlights: A fast multispectral pedestrian detection and weak segmentation system is proposed. A mask-guided mutual attention module is proposed to improve the focalizing centrality of pedestrian. A mask-guided confidence score fusion method is proposed to alleviate the inconsistency problem. Experimental results on the KAIST, CVC14 and FLIR datasets achieve state-of-the-art performance.