Task Estimation Using Latent Semantic Analysis of Visual Scenes and Spoken Words. Issue 6 (June 2014)

• Record Type: Journal Article
• Authors: Kimura, Masashi; Sawada, Shinta; Iribe, Yurie; Katsurada, Kouichi; Nitta, Tsuneo
• Is Part Of: Electronics and communications in Japan. Volume 97:Issue 6(2014:Jun.)
• Page Start: 33
• Page End: 42
• Publication Date: 2014-06
• Subjects: Electronics -- Periodicals; Telecommunication -- Periodicals; 621.6805
• DOI: 10.1002/ecj.11560
• Languages: English
• ISSNs: 1942-9533

Abstract

<abstract abstract-type="main"> <title>SUMMARY</title> <p>In this paper, we propose a task estimation method based on multiple subspaces extracted from multimodal information of image objects in visual scenes and spoken words in dialogue appearing in the same task. The multiple subspaces are obtained by using latent semantic analysis (LSA). In the proposed method, a task vector composed of spoken words and the frequencies of image‐object appearances are extracted first, and then similarities among the input task vector and reference subspaces of different tasks are compared. Experiments are conducted on the identification of game tasks. The experimental results show that the proposed method with multimodal information outperforms the method in which only the single modality of image or spoken dialogue is applied. The proposed method achieves accurate performance even if less spoken dialogue is applied.</p> </abstract>