A generic neural acoustic beamforming architecture for robust multi-channel speech processing. (November 2017)

• Record Type: Journal Article
• Authors: Heymann, Jahn; Drude, Lukas; Haeb-Umbach, Reinhold
• Is Part Of: Computer speech & language. Volume 46(2017)
• Page Start: 374
• Page End: 385
• Publication Date: 2017-11
• Subjects: Robust speech recognition -- Acoustic beamforming -- Multi-channel speech enhancement -- Deep neural network; Speech processing systems -- Periodicals; Automatic speech recognition -- Periodicals; Computers -- Periodicals; Linguistics -- Periodicals; Speech-Language Pathology -- Periodicals; Traitement automatique de la parole -- Périodiques; Reconnaissance automatique de la parole -- Périodiques; Automatic speech recognition; Speech processing systems; Electronic journals; Periodicals; 006.454
• DOI: 10.1016/j.csl.2016.11.007
• Languages: English
• ISSNs: 0885-2308

Abstract

Highlights: Generic robust beamforming approach supported by a neural network. Microphone array agnostic. No assumptions on acoustic transfer function. Robust against noise as well as reverberation. Achieves state-of-the-art ASR results for CHiME 3 and REVERB. Abstract: Acoustic beamforming can greatly improve the performance of Automatic Speech Recognition(ASR) and speech enhancement systems when multiple channels are available. We recently proposed a way to support the model-based Generalized Eigenvalue beamforming operation with a powerful neural network for spectral mask estimation. The enhancement system has a number of desirable properties. In particular, neither assumptions need to be made about the nature of the acoustic transfer function (e.g., being anechonic), nor does the array configuration need to be known. While the system has been originally developed to enhance speech in noisy environments, we show in this article that it is also effective in suppressing reverberation, thus leading to a generic trainable multi-channel speech enhancement system for robust speech processing. To support this claim, we consider two distinct datasets: The CHiME 3challenge, which features challenging real-world noise distortions, and theReverb challenge, which focuses on distortions caused by reverberation. We evaluate the system both with respect to a speech enhancement and a recognition task. For the first task we propose a new way to cope with the distortions introduced by the Generalized Eigenvalue beamformer by renormalizing the target energy for each frequency bin, and measure its effectiveness in terms of the PESQ score. For the latter we feed the enhanced signal to a strong DNN back-end and achieve state-of-the-art ASR results on both datasets. We further experiment with different network architectures for spectral mask estimation: One small feed-forward network with only one hidden layer, one Convolutional Neural Network and one bi-directional Long Short-Term Memory network, showing that even a small network is capable of delivering significant performance improvements.