Preserving the beamforming effect for spatial cue-based pseudo-binaural dereverberation of a single source. (January 2023)

• Record Type: Journal Article
• Authors: Gul, Sania; Khan, Muhammad Salman; Shah, Syed Waqar
• Is Part Of: Computer speech & language. Volume 77(2023)
• Publication Date: 2023-01
• Subjects: Beamforming -- Interaural cues -- Direct wave -- Reverberations -- Deep learning; Speech processing systems -- Periodicals; Automatic speech recognition -- Periodicals; Computers -- Periodicals; Linguistics -- Periodicals; Speech-Language Pathology -- Periodicals; Traitement automatique de la parole -- Périodiques; Reconnaissance automatique de la parole -- Périodiques; Automatic speech recognition; Speech processing systems; Electronic journals; Periodicals; 006.454
• DOI: 10.1016/j.csl.2022.101445
• Languages: English
• ISSNs: 0885-2308

Abstract

Highlights: Reverberation is a phenomenon unavoidable in enclosures, resulting in reduced intelligibility for hearing impaired and non-native listeners and even for the normal hearing listeners in noisy circumstances. It also results in degraded performance of the machine listening applications. Our proposed model has outperformed the classical signal processing dereverberation model 'weighted prediction error' in terms of cepstral distance (CEP), frequency weighted segmental signal to noise ratio (fwsegSNR) and signal-to-reverberation modulation energy ratio (SRMR) by 1.4 points, 8 dB and 0.6 dB. It has achieved better performance than the deep learning based dereverberation model by gaining 1.3 points improvement in CEP with comparable fwsegSNR, using training dataset which is almost 8 times smaller than required for that model. The proposed model also sustained its performance under relatively similar unseen acoustic conditions and at positions in the vicinity of its training position. Abstract: Reverberations are unavoidable in enclosures, resulting in reduced intelligibility for hearing impaired and non-native listeners and even for the normal hearing listeners in noisy circumstances. It also degrades the performance of machine listening applications. In this paper, we propose a novel approach of binaural dereverberation of a single speech source, using the differences in the interaural cues of the direct path signal and the reverberations. Two beamformers, spaced at an interaural distance, are used to extract the reverberations from the reverberant speech. The interaural cues generated by these reverberations and those generated by the direct path signal act as a two-class dataset, used for the training of U-Net (a deep convolutional neural network). After its training, the beamformers are removed and the trained U-Net along with the maximum likelihood estimation (MLE) algorithm is used to discriminate between the direct path cues from the reverberation cues, when the system is exposed to the interaural spectrogram of the reverberant speech signal. Our proposed model has outperformed the classical signal processing dereverberation model 'weighted prediction error' in terms of cepstral distance (CEP), frequency weighted segmental signal to noise ratio (fwsegSNR) and signal-to-reverberation modulation energy ratio (SRMR) by 1.4 points, 8 dB and 0.6 dB. It has achieved better performance than the deep learning based dereverberation model by gaining 1.3 points improvement in CEP with comparable fwsegSNR, using training dataset which is almost 8 times smaller than required for that model. The proposed model also sustained its performance under relatively similar unseen acoustic conditions and at positions in the vicinity of its training position.