Improving the understanding of spoken referring expressions through syntactic-semantic and contextual-phonetic error-correction. (November 2017)

• Record Type: Journal Article
• Authors: Zukerman, Ingrid; Partovi, Andisheh
• Is Part Of: Computer speech & language. Volume 46(2017)
• Page Start: 284
• Page End: 310
• Publication Date: 2017-11
• Subjects: Spoken language understanding -- Referring expressions -- Error correction -- Pragmatic interpretation -- Physical spaces -- Syntactic-semantic model -- Contextual-phonetic model; Speech processing systems -- Periodicals; Automatic speech recognition -- Periodicals; Computers -- Periodicals; Linguistics -- Periodicals; Speech-Language Pathology -- Periodicals; Traitement automatique de la parole -- Périodiques; Reconnaissance automatique de la parole -- Périodiques; Automatic speech recognition; Speech processing systems; Electronic journals; Periodicals; 006.454
• DOI: 10.1016/j.csl.2017.05.005
• Languages: English
• ISSNs: 0885-2308

Abstract

Highlights: We present a classifier for detecting Automatic Speech Recognition (ASR) errors. We offer a mechanism that uses shallow semantic parsing to break up the referring expressions heard by the ASR into labelled semantic segments, which are then used to set up syntactic expectations. We describe a syntactic-semantic error-correction model that decides how to modify the output of the ASR on the basis of the syntactic expectations of its semantic segments. We propose a contextual-phonetic model that re-ranks the output of a Spoken Language Understanding (SLU) system on the basis of the phonetic similarity between words mis-heard by the ASR and the contextually-valid candidate interpretations returned by the SLU system. Abstract: Despite recent advances in automatic speech recognition, one of the main stumbling blocks to the widespread adoption of Spoken Dialogue Systems is the lack of reliability of automatic speech recognizers. In this paper, we offer a two-tier error-correction process that harnesses syntactic, semantic and pragmatic information to improve the understanding of spoken referring expressions, specifically descriptions of objects in physical spaces. A syntactic-semantic tier offers generic corrections to perceived ASR errors on the basis of syntactic expectations of a semantic model, and passes the corrected texts to a language understanding system. The output of this system, which consists of pragmatic interpretations, is then refined by a contextual-phonetic tier, which prefers interpretations that are phonetically similar to the mis-heard words. Our results, obtained on a corpus of 341 referring expressions, show that syntactic-semantic error correction significantly improves interpretation performance, and contextual-phonetic refinements yield further improvements.