Simplified unsupervised image translation for semantic segmentation adaptation. (September 2020)

• Record Type: Journal Article
• Authors: Li, Rui; Cao, Wenming; Jiao, Qianfen; Wu, Si; Wong, Hau-San
• Is Part Of: Pattern recognition. Volume 105(2020:Sep.)
• Publication Date: 2020-09
• Subjects: Domain adaptation -- Image segmentation -- Image translation; Pattern perception -- Periodicals; Perception des structures -- Périodiques; Patroonherkenning; 006.4
• DOI: 10.1016/j.patcog.2020.107343
• Languages: English
• ISSNs: 0031-3203

Abstract

Highlights: A simple yet effective unsupervised image translation method (SUIT)is proposed for domain adaptation on semantic segmentation, which avoids labor-intensive pixel-wise annotation. The decoupled model design makes it moreflexible; the adaptation network be easily transplanted to other segmentation networks without repeating the adaptation process. A model average skill is developed to improve the performance in domain adaptation context. The effectiveness of our proposed model is verified on multiple synthetic-to-real adaptation benchmarks. Abstract: Image to image translation achieves superior performance with the advent of generative adversarial networks. In this paper, we propose a Simplified Unsupervised Image Translation (SUIT) model for domain adaptation on semantic segmentation. We adopt adversarial training for superior image generation, and design a novel semantic-content loss to enhance visual appearance preservation. Thus, the high-fidelity generated images with target-style can help the model generalize to the target domain. Besides, the semantic-content loss contains two components, which focus on label- and content-consistency, respectively. Both of them can be derived from existing modules of SUIT, which makes it simple yet suitable for domain adaptation on semantic segmentation tasks. Meanwhile, since the transformation network (generator) is decoupled from the segmentation network, the former can be easily transplanted to other semantic segmentation models. Extensive experimental results demonstrate that these translated images within SUIT can significantly improve performance of the model on the target domain, and our model with FCN8s-VGG16 architecture achieves around 13 percentage points improvement in terms of mIoU on multiple semantic segmentation adaptation benchmarks.