Prediction of speech intelligibility with DNN-based performance measures. (July 2022)

• Record Type: Journal Article
• Authors: Castro Martinez, Angel Mario; Spille, Constantin; Roßbach, Jana; Kollmeier, Birger; Meyer, Bernd T.
• Is Part Of: Computer speech & language. Volume 74(2022)
• Publication Date: 2022-07
• Subjects: Speech intelligibility -- Perception modeling -- Automatic speech recognition -- Speech audiometry; Speech processing systems -- Periodicals; Automatic speech recognition -- Periodicals; Computers -- Periodicals; Linguistics -- Periodicals; Speech-Language Pathology -- Periodicals; Traitement automatique de la parole -- Périodiques; Reconnaissance automatique de la parole -- Périodiques; Automatic speech recognition; Speech processing systems; Electronic journals; Periodicals; 006.454
• DOI: 10.1016/j.csl.2021.101329
• Languages: English
• ISSNs: 0885-2308

Abstract

Abstract: This paper presents a speech intelligibility model based on automatic speech recognition (ASR), combining phoneme probabilities from deep neural networks (DNN) and a performance measure that estimates the word error rate from these probabilities. This model does not require the clean speech reference nor the word labels during testing as the ASR decoding step – which finds the most likely sequence of words given phoneme posterior probabilities – is omitted. The model is evaluated via the root-mean-squared error between the predicted and observed speech reception thresholds from eight normal-hearing listeners. The recognition task consists of identifying noisy words from a German matrix sentence test. The speech material was mixed with eight noise maskers covering different modulation types, from speech-shaped stationary noise to a single-talker masker. The prediction performance is compared to five established models and an ASR-model using word labels. Two combinations of features and networks were tested. Both include temporal information either at the feature level (amplitude modulation filterbanks and a feed-forward network) or captured by the architecture (mel-spectrograms and a time-delay deep neural network, TDNN). The TDNN model is on par with the DNN while reducing the number of parameters by a factor of 37; this optimization allows parallel streams on dedicated hearing aid hardware as a forward-pass can be computed within the 10 ms of each frame. The proposed model performs almost as well as the label-based model and produces more accurate predictions than the baseline models. Graphical abstract: Highlights: New speech intelligibility model based on ASR probabilities and their estimated WER. The model does not require clean speech nor the word labels during testing. The model computes more accurate frame forward predictions within 10 ms.