DNN adaptation by automatic quality estimation of ASR hypotheses. (November 2017)

• Record Type: Journal Article
• Authors: Falavigna, Daniele; Matassoni, Marco; Jalalvand, Shahab; Negri, Matteo; Turchi, Marco
• Is Part Of: Computer speech & language. Volume 46(2017)
• Page Start: 585
• Page End: 604
• Publication Date: 2017-11
• Subjects: Deep neural networks -- DNN adaptation -- ASR quality estimation; Speech processing systems -- Periodicals; Automatic speech recognition -- Periodicals; Computers -- Periodicals; Linguistics -- Periodicals; Speech-Language Pathology -- Periodicals; Traitement automatique de la parole -- Périodiques; Reconnaissance automatique de la parole -- Périodiques; Automatic speech recognition; Speech processing systems; Electronic journals; Periodicals; 006.454
• DOI: 10.1016/j.csl.2016.11.002
• Languages: English
• ISSNs: 0885-2308

Abstract

Highlights: DNN adaptation is addressed using KLD. Manually supervised and unsupervised conditions are studied. WERs estimated with ASR Quality Estimation (in contrast with oracle WERs) used to select the adaptation material. Significant improvements w.r.t. the recent Kaldi baseline are obtained on the CHiME-3 data. Abstract: In this paper we propose to exploit the automatic Quality Estimation (QE) of ASR hypotheses to perform the unsupervised adaptation of a deep neural network modeling acoustic probabilities. Our hypothesis is that significant improvements can be achieved by: i) automatically transcribing the evaluation data we are currently trying to recognise, and ii) selecting from it a subset of "good quality" instances based on the word error rate (WER) scores predicted by a QE component. To validate this hypothesis, we run several experiments on the evaluation data sets released for the CHiME-3 challenge. First, we operate in oracle conditions in which manual transcriptions of the evaluation data are available, thus allowing us to compute the true sentence WER. In this scenario, we perform the adaptation with variable amounts of data, which are characterised by different levels of quality. Then, we move to realistic conditions in which the manual transcriptions of the evaluation data are not available. In this case, the adaptation is performed on data selected according to the WER scores predicted by a QE component. Our results indicate that: i) QE predictions allow us to closely approximate the adaptation results obtained in oracle conditions, and ii) the overall ASR performance based on the proposed QE-driven adaptation method is significantly better than the strong, most recent, CHiME-3 baseline.