Language Model Score Regularization for Speech Recognition. Issue 3 (1st May 2019)

• Record Type: Journal Article
• Authors: Zhang, Yike; Zhang, Pengyuan; Yan, Yonghong
• Is Part Of: Chinese journal of electronics. Volume 28:Issue 3(2019)
• Page Start: 604
• Page End: 609
• Publication Date: 2019-05-01
• Subjects: Speech recognition -- Language model score regularization -- Interpolation -- Bidirectional neural network; feedforward neural nets -- Markov processes -- neural nets -- probability -- recurrent neural nets -- speech recognition; fault‐tolerance -- recognition errors -- count‐based LMs -- fixed order Markov assumption -- composite model -- different order models -- skip‐gram features -- bidirectional neural networks -- N‐best lists re‐scoring show -- consistent word error rate reduction -- Feedforward neural network LMs -- Recurrent neural network LMs -- Language model score regularization -- speech recognition -- interpolated smoothing algorithms -- statistical language modeling -- sentence‐level Language model -- regularization algorithm; Electronics -- Periodicals; Electronics -- China -- Periodicals; Electronics; China; Periodicals; 621.38105
• DOI: 10.1049/cje.2019.03.015
• Languages: English
• ISSNs: 1022-4653

Abstract

Abstract : Inspired by the fact that back‐off and interpolated smoothing algorithms have significant effect on statistical language modeling, this paper proposes a sentence‐level Language model (LM) score regularization algorithm to improve the fault‐tolerance of LMs for recognition errors. The proposed algorithm is applicable to both count‐based LMs and neural network LMs. Instead of predicting the occurrence of a sequence of words under a fixed order Markov assumption, we use a composite model consisting of different order models with either n ‐gram or skip‐gram features to estimate the probability of the sequence of words. In order to simplify implementations, we derive a connection between bidirectional neural networks and the proposed algorithm. Experiments were carried out on the Switchboard corpus. Results on N ‐best lists re‐scoring show that the proposed algorithm achieves consistent word error rate reduction when it is applied to count‐based LMs, Feedforward neural network (FNN) LMs, and Recurrent neural network (RNN) LMs.