Unsupervised class-to-class translation for domain variations. (June 2023)

• Record Type: Journal Article
• Authors: Cao, Zhiyi; Wang, Wei; Huo, Lina; Niu, Shaozhang
• Is Part Of: Pattern recognition. Volume 138(2023)
• Publication Date: 2023-06
• Subjects: Contrastive learning -- Image-to-image translation -- Adversarial learning -- Image translation; Pattern perception -- Periodicals; Perception des structures -- Périodiques; Patroonherkenning; 006.4
• DOI: 10.1016/j.patcog.2023.109346
• Languages: English
• ISSNs: 0031-3203

Abstract

Highlights: We propose a new model based on conditional contrastive learning to capture the classes-related shared latent space. Since false-negative samples affect the effects of semantic clustering and class-to-class translation, we propose two condition-based contrastive learning loss functions to eliminate them. This paper first performs unsupervised semantic clustering for each domain to divide each domain into multiple classes and leverages the classification features to perform class-to-class translation. Abstract: The majority of image-to-image translation models tend to struggle in varying domain settings. For one varying domain, samples vary significantly in shape and size and have no domain labels. This paper proposes an unsupervised class-to-class translation model based on conditional contrastive learning to tackle the domain variations problem. The initial hypothesis is that the latent modalities of two varying domains are categorizable by style differences of different samples and turn the image-to-image translation problem into class-to-class translation. Firstly, unsupervised semantic clustering is performed for each domain to divide them into multiple classes and then leverage the classification features of different classes to perform class-to-class translation. Two conditional contrastive learning loss functions for each domain are proposed to perform unsupervised semantic clustering and decompose it into multiple classes. Then in the class-to-class translation stage, the classification features of different classes are employed to learn the latent modalities. The proposed model outperforms state-of-the-art baseline methods by employing the latent modalities of different classes. The sample code is available at https://github.com/c1a1o1/ucct .