An evaluation of voice conversion with neural network spectral mapping models and WaveNet vocoder. (25th November 2020)

• Record Type: Journal Article
• Authors: Tobing, Patrick Lumban; Wu, Yi-Chiao; Hayashi, Tomoki; Kobayashi, Kazuhiro; Toda, Tomoki
• Is Part Of: APSIPA transactions on signal and information processing. Volume 9(2020)
• Publication Date: 2020-11-25
• Subjects: Voice conversion, -- Neural network, -- Spectral mapping, -- WaveNet vocoder, -- Oversmoothed parameters; Signal processing -- Periodicals; 621.3822
• DOI: 10.1017/ATSIP.2020.24
• Languages: English
• ISSNs: 2048-7703

Abstract

Abstract : This paper presents an evaluation of parallel voice conversion (VC) with neural network (NN)-based statistical models for spectral mapping and waveform generation. The NN-based architectures for spectral mapping include deep NN (DNN), deep mixture density network (DMDN), and recurrent NN (RNN) models. WaveNet (WN) vocoder is employed as a high-quality NN-based waveform generation. In VC, though, owing to the oversmoothed characteristics of estimated speech parameters, quality degradation still occurs. To address this problem, we utilize post-conversion for the converted features based on direct waveform modifferential and global variance postfilter. To preserve the consistency with the post-conversion, we further propose a spectrum differential loss for the spectral modeling. The experimental results demonstrate that: (1) the RNN-based spectral modeling achieves higher accuracy with a faster convergence rate and better generalization compared to the DNN-/DMDN-based models; (2) the RNN-based spectral modeling is also capable of producing less oversmoothed spectral trajectory; (3) the use of proposed spectrum differential loss improves the performance in the same-gender conversions; and (4) the proposed post-conversion on converted features for the WN vocoder in VC yields the best performance in both naturalness and speaker similarity compared to the conventional use of WN vocoder.