Recurrent wavelet structure-preserving residual network for single image deraining. (May 2023)

• Record Type: Journal Article
• Authors: Hsu, Wei-Yen; Chang, Wei-Chi
• Is Part Of: Pattern recognition. Volume 137(2023)
• Publication Date: 2023-05
• Subjects: Single image deraining -- Recurrent wavelet residual network -- Structure preservation -- Image weighted blending; Pattern perception -- Periodicals; Perception des structures -- Périodiques; Patroonherkenning; 006.4
• DOI: 10.1016/j.patcog.2022.109294
• Languages: English
• ISSNs: 0031-3203

Abstract

Highlights: A novel Recurrent Wavelet Structure-preserving Residual Network (RWSRNet) is proposed to pay attention to both the low-frequency and high-frequency parts of rain images. We also solve the two issues: 1) Since rain streaks in rain images are often mixed with object edges and background scenes, it is difficult to separate rain from background. 2) It is also difficult to directly learn the deraining function in the image domain. The proposed method can effectively eliminate the low-frequency degradation and enrich the detailed information of high-frequency of rain images at the same time. The experimental results indicate that the proposed method achieves a good deraining effect on the both low- and high-frequency part of rain images, and has better performance in low-frequency preservation and high-frequency enhancement in comparisons with the state-of-the-art approaches on synthetic and real image datasets. Abstract: The combination of deep learning and image prior has been widely used in single image deraining since 2017. Recent studies have demonstrated an excellent deraining effect on the high-frequency part of rain images, but less attention was paid to the low-frequency part of rain images. The rain streaks remain in the low-frequency part of rain images, thus limiting the deraining effect. Since the rain streaks in rain images are often mixed with object edges and background scenes, it is challenging to separate rain from them by directly learning the deraining function in the image domain. To solve these problems, we propose a novel Recurrent Wavelet Structure-preserving Residual Network (RWSRNet), which mainly preserves and introduces the low-frequency sub-images of each level into the low-frequency rain removal sub-networks that are greatly different from the state-of-the-art approaches introducing wavelet transform. In addition, we also share the low-frequency structure information to the high-frequency sub-networks through block connection, which further enriches the detailed information, facilitates convergence, and strengthens the ability of our network to remove rain streaks in high frequency. Finally, we fuse the derained low-frequency sub-images of each level through the proposed image weighted blending module and finally reconstruct the low- and high-frequency sub-images into clean images through inverse wavelet transform recursively. The experimental results indicate that the proposed method achieves an excellent deraining effect on both low- and high-frequency parts of rain images and has better performance in low-frequency preservation and high-frequency enhancement in comparison with the state-of-the-art approaches on synthetic and real image datasets.