GAN-based image-to-friction generation for tactile simulation of fabric material. (February 2022)

• Record Type: Journal Article
• Authors: Cai, Shaoyu; Zhao, Lu; Ban, Yuki; Narumi, Takuji; Liu, Yue; Zhu, Kening
• Is Part Of: Computers & graphics. Volume 102(2022)
• Page Start: 460
• Page End: 473
• Publication Date: 2022-02
• Subjects: Supervised learning -- Generative adversarial networks (GANs) -- Haptic rendering -- Electrovibration surface -- Tactile simulation -- Fabrics; Computer graphics -- Periodicals; 006.6
• DOI: 10.1016/j.cag.2021.09.007
• Languages: English
• ISSNs: 0097-8493

Abstract

Abstract: The electrovibration tactile display could render the tactile feeling of different textured surfaces by generating the frictional force through voltage modulation. When a user is sliding his/her finger on the display surface, he/she can feel the frictional texture. However, it is not trivial to prepare and fine-tune the appropriate frictional signals for haptic design and texture simulation. In this paper, we present a deep-learning-based framework to generate the frictional signals from the textured images of fabric materials. The generated frictional signal can be used for the tactile rendering on the electrovibration tactile display. Leveraging GANs (Generative Adversarial Networks), our system could generate the displacement-based data of frictional coefficients for the tactile display to simulate the tactile feedback of different fabric materials. Our experimental results show that the proposed generative model could generate the frictional-coefficient signals visually and statistically close to the ground-truth signals. The following user studies on fabric-texture simulation show that users could not discriminate the generated and the ground-truth frictional signals being rendered on the electrovibration tactile display, suggesting the effectiveness of our deep-frictional-signal-generation model. Graphical abstract: Highlights: The deep-learning-based image-to-friction generation framework for tactile simulation of fabric material. The augmented visual-to-frictional database based on HapTex for image-to-friction generation. The technical experiment of frictional-coefficient signal generation evidencing the performance of the proposed generative model. The user-perception experiment validating the effectiveness of generated signals for tactile simulation of fabrics on the electrovibration tactile display.