Saliency detection using a deep conditional random field network. (July 2020)

• Record Type: Journal Article
• Authors: Qiu, Wenliang; Gao, Xinbo; Han, Bing
• Is Part Of: Pattern recognition. Volume 103(2020:Jul.)
• Publication Date: 2020-07
• Subjects: Saliency detection -- Conditional random field -- Convolutional neural network; Pattern perception -- Periodicals; Perception des structures -- Périodiques; Patroonherkenning; 006.4
• DOI: 10.1016/j.patcog.2020.107266
• Languages: English
• ISSNs: 0031-3203

Abstract

Highlights: Designed a multi-scale backward optimization network which can hold both rich inherent features from shallower layers and semantic features from deeper layers, then high-level features are transmitted backward to guiding low-level features. Deep CRF network is introduced to form the relationships between adjacent pixels which is crucial to improve the quality of saliency maps. All modules can be conveniently embedded into other Convolutional Neural Networks for feature and relation representation. Abstract: Saliency detection has made remarkable progress along with the development of deep learning. While how to integrate the low-level intrinsic context with high-level semantic information to keep the object boundary sharp and restrain the background noise is still a challenging problem. Many attempts on network structures and refinement strategies have been explored, such as using Conditional Random Field (CRF) to improve the accuracy of saliency map, but it is independent from the deep network and cannot be trained end-to-end. To tackle this issue, we propose a novel Deep Conditional Random Field network (DCRF) to take both deep feature and neighbor information into consideration. First, Multi-scale Feature Extraction Module (MFEM) is adopted to capture the low level texture and high level semantic features, multi-stacks of deconvolution layers are employed to improve the spatial resolution of deep layers. Then we employ Backward Optimization Module (BOM) to guide shallower layers by high-level location and shape information derived from deeper layers, which intrinsically enhance the representational capacity of low-level features. Finally, a Deep Conditional Random Field Module (DCRFM) with unary and pairwise potentials is designed to concentrate on spatial neighbor relations to obtain a compact and uniformed saliency map. Extensive experimental results on 5 datasets in terms of 6 evaluation metrics demonstrate that the proposed method achieves state-of-the-art performance.