Fourier-based augmentation with applications to domain generalization. (July 2023)

• Record Type: Journal Article
• Authors: Xu, Qinwei; Zhang, Ruipeng; Fan, Ziqing; Wang, Yanfeng; Wu, Yi-Yan; Zhang, Ya
• Is Part Of: Pattern recognition. Volume 139(2023)
• Publication Date: 2023-07
• Subjects: Domain shift -- Domain generalization -- Fourier-based augmentation -- Consistency training; Pattern perception -- Periodicals; Perception des structures -- Périodiques; Patroonherkenning; 006.4
• DOI: 10.1016/j.patcog.2023.109474
• Languages: English
• ISSNs: 0031-3203

Abstract

Highlights: We first propose an augmentation strategy based on Fourier transformation for domain generalization. The Fourier-based augmentation is implemented through amplitude mixing. We show that the Fourier-based augmentation also cooperates well with multi-view consistency training. We apply our method to three different domain generalization settings with certain modifications. Experimental results verify the effectiveness of our method. Abstract: When deployed on a new domain different from the training set, deep learning often suffers from severe performance degradation. To combat domain shift, domain adaptation and domain generalization are proposed, where the former aims at transferring knowledge from related source domains to a known target domain, while the latter is more challenging by requiring the model to generalize to unknown target domains. This paper focuses on domain generalization and introduce a novel Fourier-based perspective for it. The main idea comes from the fact that Fourier amplitude component contains low-level statistics while phase component preserves high-level semantics. We thus propose a novel Fourier-based data augmentation strategy called AmpMix by linearly interpolating the amplitudes of two images while keeping their phases unchanged, to highlight the generalizable semantics contained in phase. To make full use of Fourier-augmented samples, we further incorporate consistency training between different augmentation views and devise a Fourier-based framework for three different domain generalization settings. Extensive experiments demonstrate the effectiveness of our Fourier-based method.