Acoustic data augmentation for Mandarin-English code-switching speech recognition. (April 2020) - Long, Yanhua; Li, Yijie; Zhang, Qiaozheng; Wei, Shuang; Ye, Hong; Yang, Jichen
Highlights: Three acoustic data augmentation methods were studied for the code-switching ASR. A CS detection system was proposed to extract real code-switched speech segments. Semi-supervised and active learning were investigated to produce the CS transcriptions. The proposed approach outperforms the conventional techniques significantly. The effectiveness of synthesized code-switched utterances was very limited. Abstract: Code-switching (CS) is a multilingual phenomenon where a speaker uses different languages in an utterance or between alternating utterances. Developing large-scale datasets for training code-switching acoustic and language models is challenging and extremely expensive. In this paper, we focus on the acoustic data augmentation for the Mandarin-English CS speech recognition task. Effectiveness of conventional acoustic data augmentation approaches are examined. More importantly, we propose a CS acoustic event detection system based on the deep neural network to extract real code-switching speech segments automatically. Then, the semi-supervised and active learning techniques are investigated to generate transcriptions of these segments. Finally, code-switching speech synthesis system is introduced to further enhance the acoustic modeling. Experimental results on the OC16-CE80 data, a Mandarin-English mixlingual speech corpus, demonstrate the effectiveness of the proposed methods.