Self-supervised cycle-consistent learning for scale-arbitrary real-world single image super-resolution. (February 2023)

• Record Type: Journal Article
• Authors: Chen, Honggang; He, Xiaohai; Yang, Hong; Wu, Yuanyuan; Qing, Linbo; Sheriff, Ray E.
• Is Part Of: Expert systems with applications. Volume 212(2023)
• Publication Date: 2023-02
• Subjects: Real-world image -- Super-resolution -- Resolution-degradation -- Self-supervised cycle-consistent learning -- Arbitrary scaling factors -- Convolutional neural networks; Expert systems (Computer science) -- Periodicals; Systèmes experts (Informatique) -- Périodiques; Electronic journals; 006.33
• DOI: 10.1016/j.eswa.2022.118657
• Languages: English
• ISSNs: 0957-4174

Abstract

Abstract: Whether conventional machine learning-based or current deep neural networks-based single image super-resolution (SISR) methods, they are generally trained and validated on synthetic datasets, in which low-resolution (LR) inputs are artificially produced by degrading high-resolution (HR) images based on a hand-crafted degradation model ( e.g., bicubic downsampling). One of the main reasons for this is that it is challenging to build a realistic dataset composed of real-world LR–HR image pairs. However, a domain gap exists between synthetic and real-world data because the degradations in real scenarios are more complicated, limiting the performance in practical applications of SISR models trained with synthetic data. To address these problems, we propose a Self-supervised Cycle-consistent Learning-based Scale-Arbitrary Super-Resolution framework (SCL-SASR) for real-world images. Inspired by the Maximum a Posteriori estimation, our SCL-SASR consists of a Scale-Arbitrary Super-Resolution Network (SASRN) and an inverse Scale-Arbitrary Resolution-Degradation Network (SARDN). SARDN and SASRN restrain each other with the bidirectional cycle consistency constraints as well as image priors, making SASRN adapt to the image-specific degradation well. Meanwhile, considering the lack of targeted training images and the complexity of realistic degradations, SCL-SASR is designed to be online optimized solely with the LR input prior to the SR reconstruction. Benefitting from the flexible architecture and the self-supervised learning manner, SCL-SASR can easily super-resolve new images with arbitrary integer or non-integer scaling factors. Experiments on real-world images demonstrate the high flexibility and good applicability of SCL-SASR, which achieves better reconstruction performance than state-of-the-art self-supervised learning-based SISR methods as well as several external dataset-trained SISR models. Highlights: We propose a self-learning-based scale-arbitrary SR method for real-world images. Scale-arbitrary SR and resolution-degradation networks are jointly optimized. The cycle consistency, bicubic interpolation, and total variation losses are adopted. We enable a parameter to adjust perceptual quality to satisfy users' preferences. Extensive experiments are conducted to demonstrate the effectiveness of our method.