A dilemma of ground truth in noisy speech separation and an approach to lessen the impact of imperfect training data. (January 2023) - Maciejewski, Matthew; Shi, Jing; Watanabe, Shinji; Khudanpur, Sanjeev
Abstract: As the performance of single-channel speech separation systems has improved, there has been a shift in the research community towards tackling more challenging conditions that are more representative of many real-world applications, including the addition of noise and reverberation. The need for ground truth in training state-of-the-art separation systems leads to a requirement of training on artificial mixtures, where single-speaker recordings are summed digitally. However, this leads to two separate approaches for creating noisy mixtures: one in which noise has been artificially added, maintaining perfect ground truth information, and one in which the noise is already present in the single-speaker recordings, allowing for in-domain training. In this work, we document a severe negative impact in both training and evaluation of models in the latter paradigm. We provide an explanation for this – the implicit task of separating noise – and propose an improved training objective that allows errors resulting from failing to separate noise to be minimized. Highlights: Describes the two approaches used for creation of noisy condition datasets for single-channel speech separation. Demonstrates a gap in separation performance between models trained using different types of noisy data. Proposes a new objective function for more effective training of separation systems on noisy ground truth data.