A dilemma of ground truth in noisy speech separation and an approach to lessen the impact of imperfect training data. (January 2023)

• Record Type: Journal Article
• Authors: Maciejewski, Matthew; Shi, Jing; Watanabe, Shinji; Khudanpur, Sanjeev
• Is Part Of: Computer speech & language. Volume 77(2023)
• Publication Date: 2023-01
• Subjects: Speech separation -- Noisy speech -- Deep learning; Speech processing systems -- Periodicals; Automatic speech recognition -- Periodicals; Computers -- Periodicals; Linguistics -- Periodicals; Speech-Language Pathology -- Periodicals; Traitement automatique de la parole -- Périodiques; Reconnaissance automatique de la parole -- Périodiques; Automatic speech recognition; Speech processing systems; Electronic journals; Periodicals; 006.454
• DOI: 10.1016/j.csl.2022.101410
• Languages: English
• ISSNs: 0885-2308

Abstract

Abstract: As the performance of single-channel speech separation systems has improved, there has been a shift in the research community towards tackling more challenging conditions that are more representative of many real-world applications, including the addition of noise and reverberation. The need for ground truth in training state-of-the-art separation systems leads to a requirement of training on artificial mixtures, where single-speaker recordings are summed digitally. However, this leads to two separate approaches for creating noisy mixtures: one in which noise has been artificially added, maintaining perfect ground truth information, and one in which the noise is already present in the single-speaker recordings, allowing for in-domain training. In this work, we document a severe negative impact in both training and evaluation of models in the latter paradigm. We provide an explanation for this – the implicit task of separating noise – and propose an improved training objective that allows errors resulting from failing to separate noise to be minimized. Highlights: Describes the two approaches used for creation of noisy condition datasets for single-channel speech separation. Demonstrates a gap in separation performance between models trained using different types of noisy data. Proposes a new objective function for more effective training of separation systems on noisy ground truth data.