The residual generator: An improved divergence minimization framework for GAN. (January 2022)

• Record Type: Journal Article
• Authors: Gnanha, Aurele Tohokantche; Cao, Wenming; Mao, Xudong; Wu, Si; Wong, Hau-San; Li, Qing
• Is Part Of: Pattern recognition. Volume 121(2022)
• Publication Date: 2022-01
• Subjects: Generative adversarial networks -- Image synthesis -- Deep learning; Pattern perception -- Periodicals; Perception des structures -- Périodiques; Patroonherkenning; 006.4
• DOI: 10.1016/j.patcog.2021.108222
• Languages: English
• ISSNs: 0031-3203

Abstract

Highlights: We propose a residual generator (Rg-GAN) served as a better approximation divergence minimization framework for GAN, and prove that residual generator for standard and least-squares GAN are equivalent to the minimization of reverse-KL and a new instance of f-divergence, respectively. We prove that Rg-GAN can be reduced to IPMs based GAN and bridge the gap between IPMs and f-divergence. We propose a new loss function for the discriminator of Rg-GAN that manifests a better discriminative property and therefore improved on Rg-GAN generalisation ability. We conduct experiments on multiple benchmark data sets and demonstrate that our proposed framework can mitigate the mode collapse issue and facilitate GAN to generate higher-quality images with negligible additional computation cost. Abstract: GAN is a generative modelling framework which has been proven as able to minimise various types of divergence measures under an optimal discriminator. However, there is a gap between the loss function of GAN used in theory and in practice. In theory, the proof of the Jensen divergence minimisation involves the min-max criterion, but in practice the non-saturating criterion is instead used to avoid gradient vanishing. We argue that the formulation of divergence minimization via GAN is biased and may yield a poor convergence of the algorithm. In this paper, we propose the Residual Generator for GAN (Rg-GAN), which is inspired by the closed-loop control theory, to bridge the gap between theory and practice. Rg-GAN minimizes the residual between the loss of the generated data to be real and the loss of the generated data to be fake from the perspective of the discriminator. In this setting, the loss terms of the generator depend only on the generated data and therefore contribute to the optimisation of the model. We formulate the residual generator for standard GAN and least-squares GAN and show that they are equivalent to the minimisation of reverse-KL divergence and a novel instance of f-divergence, respectively. Furthermore, we prove that Rg-GAN can be reduced to Integral Probability Metrics (IPMs) GANs (e.g., Wasserstein GAN) and bridge the gap between IPMs and f-divergence. Additionally, we further improve on Rg-GAN by proposing a loss function for the discriminator that has a better discrimination ability. Experiments on synthetic and natural images data sets show that Rg-GAN is robust to mode collapse, and improves the generation quality of GAN in terms of FID and IS scores.