Pansharpening approach via two-stream detail injection based on relativistic generative adversarial networks. (February 2022)

• Record Type: Journal Article
• Authors: Benzenati, Tayeb; Kessentini, Yousri; Kallel, Abdelaziz
• Is Part Of: Expert systems with applications. Volume 188(2022)
• Publication Date: 2022-02
• Subjects: Pansharpening -- CNN -- Detail injection -- Relativistic GAN; Expert systems (Computer science) -- Periodicals; Systèmes experts (Informatique) -- Périodiques; Electronic journals; 006.33
• DOI: 10.1016/j.eswa.2021.115996
• Languages: English
• ISSNs: 0957-4174

Abstract

Abstract: The goal of pansharpening technique is to reconstruct a high resolution spectral image (HRMS) from a low spatial resolution multispectral image (MS) and a high spatial resolution single band panchromatic (PAN) image. As the efficacy of such a technique relies upon its capability to reinforce the spatial information of the MS image while conserving its spectral signature, in this paper, we propose DI-GAN, an effective Detail Injection Generative Adversarial Network for remote sensing image pansharpening. DI-GAN employs a deep two-stream Convolutional Neural Network (CNN) generator model to combine the spatial information available on the PAN image and the spectral characteristic belonging to the MS image, in order to predict the high-frequency detail to be accurately injected into the MS image to reconstruct the desired HRMS. An original loss function is particularly proposed to encourage the network to predict only high-frequency detail. DI-GAN incorporates in parallel a Relativistic Discriminator that allows the pansharpening products to be more realistic. Experimental results at degraded and full scale on three different datasets were showed that the proposed fusion approach can produce superior pansharpening performances in terms of spatial and spectral fidelities with respect to the literature techniques, including CNN- and GAN-based methods. Highlights: We propose a new pan-sharpening technique based on relativistic generative adversarial networks. It enhances the spatial quality by injecting the high-frequency detail into the MS image. The generative behavior is improved by exploiting a Relativistic average Discriminator. An effective loss makes the fusion product more realistic with high spectral fidelity. Our technique yields very competitive visual and quantitative fusion results.