On the relevance of auditory-based Gabor features for deep learning in robust speech recognition. (September 2017)

• Record Type: Journal Article
• Authors: Castro Martinez, Angel Mario; Mallidi, Sri Harish; Meyer, Bernd T.
• Is Part Of: Computer speech & language. Volume 45(2017)
• Page Start: 21
• Page End: 38
• Publication Date: 2017-09
• Subjects: Auditory features -- Spectro-temporal processing -- Deep neural networks -- Automatic speech recognition; Speech processing systems -- Periodicals; Automatic speech recognition -- Periodicals; Computers -- Periodicals; Linguistics -- Periodicals; Speech-Language Pathology -- Periodicals; Traitement automatique de la parole -- Périodiques; Reconnaissance automatique de la parole -- Périodiques; Automatic speech recognition; Speech processing systems; Electronic journals; Periodicals; 006.454
• DOI: 10.1016/j.csl.2017.02.006
• Languages: English
• ISSNs: 0885-2308

Abstract

Highlights: DNN-based speech recognition greatly benefits from spectro-temporal Gabor features. Gabor filters with high temporal modulation encode the most relevant information. A measure of phoneme similarity is proposed to quantify class separability. This metric is used to explain the improved results on phoneme level. Graphical abstract: Abstract: Previous studies support the idea of merging auditory-based Gabor features with deep learning architectures to achieve robust automatic speech recognition, however, the cause behind the gain of such combination is still unknown. We believe these representations provide the deep learning decoder with more discriminable cues. Our aim with this paper is to validate this hypothesis by performing experiments with three different recognition tasks (Aurora 4, CHiME 2 and CHiME 3) and assess the discriminability of the information encoded by Gabor filterbank features. Additionally, to identify the contribution of low, medium and high temporal modulation frequencies subsets of the Gabor filterbank were used as features (dubbed LTM, MTM and HTM, respectively). With temporal modulation frequencies between 16 and 25 Hz, HTM consistently outperformed the remaining ones in every condition, highlighting the robustness of these representations against channel distortions, low signal-to-noise ratios and acoustically challenging real-life scenarios with relative improvements from 11 to 56% against a Mel-filterbank-DNN baseline. To explain the results, a measure of similarity between phoneme classes from DNN activations is proposed and linked to their acoustic properties. We find this measure to be consistent with the observed error rates and highlight specific differences on phoneme level to pinpoint the benefit of the proposed features.