A Novel Unsupervised domain adaptation method for inertia-Trajectory translation of in-air handwriting. (August 2021)

• Record Type: Journal Article
• Authors: Xu, Songbin; Xue, Yang; Zhang, Xin; Jin, Lianwen
• Is Part Of: Pattern recognition. Volume 116(2021)
• Publication Date: 2021-08
• Subjects: In-air handwriting -- Bi-directional inertia-Trajectory translation -- Unsupervised domain adaptation -- Latent-level adversarial learning; Pattern perception -- Periodicals; Perception des structures -- Périodiques; Patroonherkenning; 006.4
• DOI: 10.1016/j.patcog.2021.107939
• Languages: English
• ISSNs: 0031-3203

Abstract

Highlights: We propose a novel Air Writing Translator for inertia trajectory transla tion of in air handwriti ng. To our best knowledge, this is the f i rst attempt to address unsupervised inertia trajectory translation as a domain adap tation task in IAHR. We propose a triple loss function to train the network: adversarial loss, classification loss and reconstruction loss. The combination of adversarial loss and latent classification loss in latent space aims to guide the semantic consistency of feature level. Coupled with reconstruction loss, the triple loss ensures that the model can learn the style information of the target We propose a unique design of Conv and GRU (gated recurrent unit) to handle input time series of arbitrary length. In addition, this design makes it possible to translate between inertial signals and trajectory signals of different sampling rat Air-Writing Translator is trained in an unsupervised manner, with no need for pair-wise data. On one hand, this training method ensures that our model can learn to make latent representations of the two domains closer from the perspective of probability distribution, rather than simply matching samples. On the other hand, it solves the problem of lack of paired data in practical applications. Abstract: As a new method of human-computer interaction, inertial sensor-based in-air handwriting can provide natural and unconstrained interaction to express more complex and rich information in 3D space. However, most of the existing literature is mainly focused on in-air handwriting recognition (IAHR), which makes these works suffer from the poor readability of inertial signals and the lack of labeled samples. To address these two problems, we use an unsupervised domain adaptation method to recover the trajectory of inertial signals and generate inertial samples using handwritten trajectories. In this paper, we propose an Air-Writing Translator model to learn the bi-directional translation between trajectory domain and inertial domain in the absence of paired inertial and trajectory samples. Through latent-level adversarial learning and latent classification loss, the proposed model learns to extract domain-invariant features between the inertial signal and the trajectory while preserving semantic consistency during the translation across the two domains. In addition, the proposed framework can accept inputs of arbitrary length and translate between different sampling rates. Experiments on two public datasets, 6DMG (in-air handwriting dataset) and CT (handwritten trajectory dataset), are conducted and the results demonstrate that the proposed model can achieve reliable translation between inertial domain and trajectory domain. Empirically, our method also yields the best results in comparison to the state-of-the-art methods for IAHR.