Combining spectral and temporal modification techniques for speech intelligibility enhancement. (May 2019) - Cooke, Martin; Aubanel, Vincent; García Lecumberri, María Luisa
Highlights: Spectral and temporal modification techniques combine synergistically. For Spanish sentences, error rates are reduced by a factor of 3 compared to unmodified speech. All phonemes benefit from spectral and temporal modification. A glimpsing model predicts listener performance with a correlation of 0.96. Cochlear-Scaled Entropy does not improve the performance of a retiming algorithm. Abstract: Modifying clean speech prior to output in noisy conditions can lead to substantial intelligibility gains. Most algorithms operate by redistributing energy across the signal, leaving the timing of the underlying speech sounds intact. Other techniques do alter the timing of speech relative to the masker. Both classes of approach – spectral and temporal – lead to a reduction in energetic masking. The current study examines how their combination affects intelligibility. Arguments can be made for both synergy and redundancy, and the presence of distortions introduced by both spectral and temporal approaches might even lead to an antagonistic combination. A cohort of native Spanish listeners identified keywords in sentences in unmodified form and following spectral, temporal and spectro-temporal modification, in the presence of a fluctuating masker. Errors in the spectro-temporal condition were substantially lower than following spectral or temporal modification alone, with a three-fold reduction compared to unmodified speech. Spectro-temporal gains were observed for all phonemes. A glimpse-based model of energetic masking incorporating speech rate changes predicts intelligibility ( r = . 96 ), and a glimpsing analysis provides further insights into the distinct mechanisms through which spectral and temporal approaches lead to a release from energetic masking.