Single-Image super-resolution - When model adaptation matters. (August 2021)

• Record Type: Journal Article
• Authors: Liang, Yudong; Timofte, Radu; Wang, Jinjun; Zhou, Sanping; Gong, Yihong; Zheng, Nanning
• Is Part Of: Pattern recognition. Volume 116(2021)
• Publication Date: 2021-08
• Subjects: Internal prior -- Model adaptation -- Deep convolutional neural network -- Projection skip connection; Pattern perception -- Periodicals; Perception des structures -- Périodiques; Patroonherkenning; 006.4
• DOI: 10.1016/j.patcog.2021.107931
• Languages: English
• ISSNs: 0031-3203

Abstract

Highlights: In this paper, we propose a variation of deep residual convolutional neural networks with robustness and efficiency in both learning and testing. More importantly, we propose multiple strategies for model adaptation to the internal contents of the lowresolution input image and analyze their strong points and weaknesses. Our adaptation especially favors images with repetitive structures or high resolutions. We hope to arouse the interests of communities in focusing on internal priors, which are limited but have been proved effective and highly relevant. Abstract: In recent years, impressive advances have been made in single-image super-resolution. Deep learning is behind much of this success. Deep(er) architecture design and external prior modeling are the key ingredients. The internal contents of the low-resolution input image are neglected with deep modeling, despite earlier works that show the power of using such internal priors. In this paper, we propose a variation of deep residual convolutional neural networks, which has been carefully designed for robustness and efficiency in both learning and testing. Moreover, we propose multiple strategies for model adaptation to the internal contents of the low-resolution input image and analyze their strong points and weaknesses. By trading runtime and using internal priors, we achieve improvements from 0.1 to 0.3 dB PSNR over the reported results on standard datasets. Our adaptation especially favors images with repetitive structures or high resolutions. It indicates a more practical usage when our adaption approach applies to sequences or videos in which adjacent frames are strongly correlated in their contents. Moreover, the approach can be combined with other simple techniques, such as back-projection and enhanced prediction, to realize further improvements.