Defect attention template generation cycleGAN for weakly supervised surface defect segmentation. (March 2022)

• Record Type: Journal Article
• Authors: Niu, Shuanlong; Li, Bin; Wang, Xinggang; He, Songping; Peng, Yaru
• Is Part Of: Pattern recognition. Volume 123(2022)
• Publication Date: 2022-03
• Subjects: Weakly supervised learning -- Defect detection -- Image segmentation -- Generative adversarial network (GAN) -- Attention model; Pattern perception -- Periodicals; Perception des structures -- Périodiques; Patroonherkenning; 006.4
• DOI: 10.1016/j.patcog.2021.108396
• Languages: English
• ISSNs: 0031-3203

Abstract

Highlights: A weakly supervised defect segmentation method that does not require pixel-level annotations of defects for training is proposed based on the dynamic templates generated by an improved Cycle-GAN, which can achieve significantly higher performance than other weakly supervised methods and have competitive performance with supervised methods. A novel defect attention module is proposed in the discriminator of Cycle-GAN to better eliminate defects with weak signals, such as small region and low contrast, for more accurate template images. A defect consistency loss is proposed by adding SSIM to the L1 loss based on grayscale and texture, respectively, for better modeling the inner structure of defects. The proposed method has been successfully applied at an industrial site for commutator surface defect detection and has exhibited excellent detection accuracy and significantly reduced manual labeling costs. The proposed method can also be employed as a semiautomatic annotation tool combined with active learning. Abstract: Surface defect segmentation is very important for the quality inspection of industrial production and is an important pattern recognition problem. Although deep learning (DL) has achieved remarkable results in surface defect segmentation, most of these results have been obtained by using massive images with pixel-level annotations, which are difficult to obtain at industrial sites. This paper proposes a weakly supervised defect segmentation method based on the dynamic templates generated by an improved cycle-consistent generative adversarial network (CycleGAN) trained by image-level annotations. To generate better templates for defects with weak signals, we propose a defect attention module by applying the defect residual for the discriminator to strengthen the elimination of defect regions and suppress changes in the background. A defect cycle-consistent loss is designed by adding structural similarity (SSIM) to the original L1 loss to include the grayscale and structural features; the proposed loss can better model the inner structure of defects. After obtaining the defect-free template, a defect segmentation map can easily be obtained through a simple image comparison and threshold segmentation. Experiments show that the proposed method is both efficient and effective, significantly outperforms other weakly supervised methods, and achieves performance that is comparable or even superior to that of supervised methods on three industrial datasets (intersection over union (IoU) on the DAGM 2007, KSD and CCSD datasets of 78.28%, 59.43%, and 68.83%, respectively). The proposed method can also be employed as a semiautomatic annotation tool combined with active learning.