Using Highway Connections to Enable Deep Small‐footprint LSTM‐RNNs for Speech Recognition. Issue 1 (1st January 2019)

• Record Type: Journal Article
• Authors: CHENG, Gaofeng; LI, Xin; YAN, Yonghong
• Is Part Of: Chinese journal of electronics. Volume 28:Issue 1(2019)
• Page Start: 107
• Page End: 112
• Publication Date: 2019-01-01
• Subjects: Long short‐term memory -- Highway connections -- Small‐footprint -- Speech recognition.; learning (artificial intelligence) -- recurrent neural nets -- speech recognition; highway connections -- deep small‐footprint LSTM‐RNNs -- short‐term memory RNNs -- Automatic speech recognition field -- time‐sequence modeling tasks -- deep LSTM‐RNNs -- memory cells -- small‐footprint highway LSTM‐RNNs -- 3‐layer LSTM‐RNNs -- small‐footprint HLSTM‐RNNs -- acoustic model -- switchboard; Electronics -- Periodicals; Electronics -- China -- Periodicals; Electronics; China; Periodicals; 621.38105
• DOI: 10.1049/cje.2018.11.008
• Languages: English
• ISSNs: 1022-4653

Abstract

Abstract : Long short‐term memory RNNs (LSTMRNNs) have shown great success in the Automatic speech recognition (ASR) field and have become the state‐ofthe‐ art acoustic model for time‐sequence modeling tasks. However, it is still difficult to train deep LSTM‐RNNs while keeping the parameter number small. We use the highway connections between memory cells in adjacent layers to train a small‐footprint highway LSTM‐RNNs (HLSTM‐RNNs), which are deeper and thinner compared to conventional LSTM‐RNNs. The experiments on the Switchboard (SWBD) indicate that we can train thinner and deeper HLSTM‐RNNs with a smaller parameter number than the conventional 3‐layer LSTM‐RNNs and a lower Word error rate (WER) than the conventional one. Compared with the counterparts of small‐footprint LSTMRNNs, the small‐footprint HLSTM‐RNNs show greater reduction in WER.