Situated language understanding for a spoken dialog system within vehicles. (November 2015)

• Record Type: Journal Article
• Authors: Misu, Teruhisa; Raux, Antoine; Gupta, Rakesh; Lane, Ian
• Is Part Of: Computer speech & language. Volume 34(2015)
• Page Start: 186
• Page End: 200
• Publication Date: 2015-11
• Subjects: Spoken dialog systems -- Situated dialog -- Language understanding -- Reference resolution; Speech processing systems -- Periodicals; Automatic speech recognition -- Periodicals; Computers -- Periodicals; Linguistics -- Periodicals; Speech-Language Pathology -- Periodicals; Traitement automatique de la parole -- Périodiques; Reconnaissance automatique de la parole -- Périodiques; Automatic speech recognition; Speech processing systems; Electronic journals; Periodicals; 006.454
• DOI: 10.1016/j.csl.2015.02.002
• Languages: English
• ISSNs: 0885-2308

Abstract

Abstract : Highlights: We implemented and analyzed issues in situated language understanding in moving car. We analyzed timing of utterances, spatial relationships between the car and targets. Our algorithms improved the target identification rate by 24.1%. Abstract: In this paper, we address issues in situated language understanding in a moving car, which has the additional challenge of being a rapidly changing environment. More specifically, we propose methods for understanding user queries regarding specific target buildings in their surroundings. Unlike previous studies on physically situated interactions, such as interactions with mobile robots, the task at hand is very time sensitive because the spatial relationship between the car and target changes while the user is speaking. We collected situated utterances from drivers using our research system called Townsurfer, which was embedded in a real vehicle. Based on this data, we analyzed the timing of user queries, the spatial relationships between the car and the targets, the head pose of the user, and linguistic cues. Based on this analysis, we further propose methods to optimize timing and spatial distances and to make use of linguistic cues. Finally, we demonstrate that our algorithms improved the target identification rate by 24.1% absolute.