Unsupervised language model adaptation using LDA-based mixture models and latent semantic marginals. (January 2015)

• Record Type: Journal Article
• Authors: Haidar, Md. Akmal; O'Shaughnessy, Douglas
• Is Part Of: Computer speech & language. Volume 29(2015)
• Page Start: 20
• Page End: 31
• Publication Date: 2015-01
• Subjects: Language model -- Topic model -- Mixture model -- Speech recognition -- Minimum discriminant information; Speech processing systems -- Periodicals; Automatic speech recognition -- Periodicals; Computers -- Periodicals; Linguistics -- Periodicals; Speech-Language Pathology -- Periodicals; Traitement automatique de la parole -- Périodiques; Reconnaissance automatique de la parole -- Périodiques; Automatic speech recognition; Speech processing systems; Electronic journals; Periodicals; 006.454
• DOI: 10.1016/j.csl.2014.06.002
• Languages: English
• ISSNs: 0885-2308

Abstract

Abstract : Highlights: Unsupervised language model adaptation approaches for speech recognition. A hard-clustering approach is applied into latent Dirichlet allocation (LDA) model. Tri-gram topic models are created and mixed to form an adapted tri-gram model. The adapted tri-gram model is interpolated with a background tri-gram model. The above models are modified using unigram latent semantic marginals. Abstract: In this paper, we present unsupervised language model (LM) adaptation approaches using latent Dirichlet allocation (LDA) and latent semantic marginals (LSM). The LSM is the unigram probability distribution over words that are calculated using LDA-adapted unigram models. The LDA model is used to extract topic information from a training corpus in an unsupervised manner. The LDA model yields a document–topic matrix that describes the number of words assigned to topics for the documents. A hard-clustering method is applied on the document–topic matrix of the LDA model to form topics. An adapted model is created by using a weighted combination of the n -gram topic models. The stand-alone adapted model outperforms the background model. The interpolation of the background model and the adapted model gives further improvement. We modify the above models using the LSM. The LSM is used to form a new adapted model by using the minimum discriminant information (MDI) adaptation approach called unigram scaling, which minimizes the distance between the new adapted model and the other model. The unigram scaling of the adapted model using LSM yields better results over a conventional unigram scaling approach. The unigram scaling of the interpolation of the background and the adapted model using the LSM outperform the background model, the unigram scaling of the background model, the unigram scaling of the adapted model, and the interpolation of the background and the adapted models respectively. We perform experiments using the '87–89 Wall Street Journal (WSJ) corpus incorporating a multi-pass continuous speech recognition (CSR) system. In the first pass, we used the background n -gram language model for lattice generation and then we apply the LM adaptation approaches for lattice rescoring in the second pass.