Semi-supervised student-teacher learning for single image super-resolution. (January 2022)

• Record Type: Journal Article
• Authors: Wang, Lin; Yoon, Kuk-Jin
• Is Part Of: Pattern recognition. Volume 121(2022)
• Publication Date: 2022-01
• Subjects: Semi-supervised learning -- Image super-resolution -- Student-teacher model -- Adversarial learning; Pattern perception -- Periodicals; Perception des structures -- Périodiques; Patroonherkenning; 006.4
• DOI: 10.1016/j.patcog.2021.108206
• Languages: English
• ISSNs: 0031-3203

Abstract

Highlights: We propose a deep learning-based semi-supervised approach for SISR. We build our framework via adversarial learning. To better facilitate the learning of unlabelled data, we propose a student-teacher (S-T) model to transfer the knowledge from supervised learning (teacher) to the unsupervised learning (student). The S-T model is based on partial weight-sharing of dual discriminators and a pair matching network playing two roles: cycle consistency and 'latent discriminator' for better learning of unlabelled data. We propose a new SR network structure to better learn the non-local features from LR images via channel in channel and spatial in spatial mechanisms. We demonstrate that our method outperforms some purely supervised and unsupervised methods on various experimental settings. Abstract: Most existing approaches for single image super-resolution (SISR) resort to quality low-high resolution (LR-HR) pairs and available degradation kernels to train networks for a specific task in hand in a fully supervised manner. Labeled data used for training are, however, usually limited in terms of the quantity and the diversity degradation kernels. The learned SR networks with one degradation kernel ( e.g ., bicubic) do not generalize well and their performance sharply deteriorates on other kernels ( e.g ., blurred or noise). In this paper, we address the critical challenge for SISR: limited labeled LR images and degradation kernels. We propose a novel S emi-supervised S tudent-T eacher S uper-R esolution approach called S 2 TSR that super-resolves both labelled and unlabeled LR images via adversarial learning. To better exploit the information from labeled LR images, we propose a student-teacher framework (S-T) via knowledge transfer from supervised learning (T) to unsupervised learning (S). Specifically, the S-T knowledge transfer is based on a shared SR network, partial weight sharing of dual discriminators, and a pair matching network which also plays as a 'latent discriminator'. Lastly, to learn better features from the limited labeled LR images, we propose a new SR network via non-local and attention mechanisms. Experiments demonstrate that our approach substantially improves unsupervised methods and performs favorably over fully supervised methods.