Lightweight Semi-supervised Network for Single Image Rain Removal. (May 2023)

• Record Type: Journal Article
• Authors: Jiang, Nanfeng; Luo, Jiawei; Lin, Junhong; Chen, Weiling; Zhao, Tiesong
• Is Part Of: Pattern recognition. Volume 137(2023)
• Publication Date: 2023-05
• Subjects: Rain Removal -- Image Processing -- Lightweight Network -- Semi-supervised Learning; Pattern perception -- Periodicals; Perception des structures -- Périodiques; Patroonherkenning; 006.4
• DOI: 10.1016/j.patcog.2022.109277
• Languages: English
• ISSNs: 0031-3203

Abstract

Highlights: We utilize the semi-supervised strategy to alleviate the gaps between synthetic and real-world images. To reduce model size, we adopt recursive strategy to reuse model parameters. In addition, the proposed LSNet is capable of learning rich hierarchical features at a coarse-to-fine image residual estimation and improve model robustness. Our semi-supervised framework has a supervised branch and an unsupervised branch. On both branches, we design a cascaded sub-network, which effectively handles different types of rainy images via a multi-stage processing. On the supervised branch, we also introduce perceptual loss to improve image qualities. To preserve structures and details, we use Total Variation (TV) loss to constrain the unsupervised branch Our proposed LSNet has the advantages of strong robustness and efficient calculation. Experimental results show that the proposed model achieves an excellent balance among visual quality, inference speed and model parameters. As demonstrated by extensive experiments, our model promotes the performances of real-world applications, e.g., object detction. Abstract: Deep learning technologies have shown their advantages in Single Image Rain Removal (SIRR) tasks. However, the derained results of most methods are limited to some challenges. First, due to the lack of real-world rainy/clean image pairs, many methods seriously rely on the labeled synthetic training images and will not effectively remove complex rain streaks in real-world scenarios. Second, most existing SIRR models require high computing power, which considerably limits their real-world applications. To address these issues, we propose a Lightweight Semi-supervised Network (LSNet) for SIRR. Our LSNet utilizes a compact semi-supervised framework to improve generalization ability in real-world rainy images removal. Meanwhile, in our semi-supervised framework, we also design a cascaded sub-network, which progressively removes complex rain streaks via a multi-stage manner. Specially, the multi-stage manner is based on a series of cascaded blocks, where we conduct recursive learning strategy to reduce model parameters. Extensive experimental results demonstrate that our method achieves comparable performance to the state-of-the-arts while has fewer parameters.