Multi-Channel Speech Enhancement and Amplitude Modulation Analysis for Noise Robust Automatic Speech Recognition. (November 2017)

• Record Type: Journal Article
• Authors: Moritz, Niko; Adiloğlu, Kamil; Anemüller, Jörn; Goetze, Stefan; Kollmeier, Birger
• Is Part Of: Computer speech & language. Volume 46(2017)
• Page Start: 558
• Page End: 573
• Publication Date: 2017-11
• Subjects: Speech enhancement -- Non-negative matrix factorization -- Feature extraction -- Modulation frequency analysis -- CHiME -- Amplitude modulation filter bank; Speech processing systems -- Periodicals; Automatic speech recognition -- Periodicals; Computers -- Periodicals; Linguistics -- Periodicals; Speech-Language Pathology -- Periodicals; Traitement automatique de la parole -- Périodiques; Reconnaissance automatique de la parole -- Périodiques; Automatic speech recognition; Speech processing systems; Electronic journals; Periodicals; 006.454
• DOI: 10.1016/j.csl.2016.11.004
• Languages: English
• ISSNs: 0885-2308

Abstract

Highlights: An elaborate ASR system is proposed that outperforms the best results of the 3rd CHiME challenge. Amplitude modulation filter bank (AMFB) features demonstrate enhanced ASR robustness in comparison to the commonly used frame splicing approach. A microphone failure detection method is proposed to detect erroneous recordings of a multichannel recording device. Qualities of two multi-channel speech enhancement techniques, which are a MVDR and NMF-based method, are emphasized by system combination using minimum Bayes risk (MBR) decoding. Abstract: The paper describes a system for automatic speech recognition (ASR) that is benchmarked with data of the 3rd CHiME challenge, a dataset comprising distant microphone recordings of noisy acoustic scenes in public environments. The proposed ASR system employs various methods to increase recognition accuracy and noise robustness. Two different multi-channel speech enhancement techniques are used to eliminate interfering sounds in the audio stream. One speech enhancement method aims at separating the target speaker's voice from background sources based on non-negative matrix factorization (NMF) using variational Bayesian (VB) inference to estimate NMF parameters. The second technique is based on a time-varying minimum variance distortionless response (MVDR) beamformer that uses spatial information to suppress sound signals not arriving from a desired direction. Prior to speech enhancement, a microphone channel failure detector is applied that is based on cross-comparing channels using a modulation-spectral representation of the speech signal. ASR feature extraction employs the amplitude modulation filter bank (AMFB) that implicates prior information of speech to analyze its temporal dynamics. AMFBs outperform the commonly used frame splicing technique of filter bank features in conjunction with a deep neural network (DNN) based ASR system, which denotes an equivalent data-driven approach to extract modulation-spectral information. In addition, features are speaker adapted, a recurrent neural network (RNN) is employed for language modeling, and hypotheses of different ASR systems are combined to further enhance the recognition accuracy. The proposed ASR system achieves an absolute word error rate (WER) of 5.67% on the real evaluation test data, which is 0.16% lower compared to the best score reported within the 3rd CHiME challenge.