A Voice Cloning Method Based on the Improved HiFi-GAN Model. (11th October 2022)

• Record Type: Journal Article
• Authors: Qiu, Zeyu; Tang, Jun; Zhang, Yaxin; Li, Jiaxin; Bai, Xishan
• Other Names: Cao Wenming Academic Editor.
• Is Part Of: Computational intelligence and neuroscience. Volume 2022(2022)
• Publication Date: 2022-10-11
• Subjects: Neurosciences -- Data processing -- Periodicals; Computational intelligence -- Periodicals; Computational neuroscience -- Periodicals; 612.80285
• DOI: 10.1155/2022/6707304
• Languages: English
• ISSNs: 1687-5265

Abstract

Abstract : With the aim of adapting a source Text to Speech (TTS) model to synthesize a personal voice by using a few speech samples from the target speaker, voice cloning provides a specific TTS service. Although the Tacotron 2-based multi-speaker TTS system can implement voice cloning by introducing a d-vector into the speaker encoder, the speaker characteristics described by the d-vector cannot allow for the voice information of the entire utterance. This affects the similarity of voice cloning. As a vocoder, WaveNet sacrifices speech generation speed. To balance the relationship between model parameters, inference speed, and voice quality, a voice cloning method based on improved HiFi-GAN has been proposed in this paper. (1) To improve the feature representation ability of the speaker encoder, the x-vector is used as the embedding vector that can characterize the target speaker. (2) To improve the performance of the HiFi-GAN vocoder, the input Mel spectrum is processed by a competitive multiscale convolution strategy. (3) The one-dimensional depth-wise separable convolution is used to replace all standard one-dimensional convolutions, significantly reducing the model parameters and increasing the inference speed. The improved HiFi-GAN model remarkably reduces the number of vocoder model parameters by about 68.58% and boosts the model's inference speed. The inference speed on the GPU and CPU has increased by 11.84% and 30.99%, respectively. Voice quality has also been marginally improved as MOS increased by 0.13 and PESQ increased by 0.11. The improved HiFi-GAN model exhibits outstanding performance and remarkable compatibility in the voice cloning task. Combined with the x-vector embedding, the proposed model achieves the highest score of all the models and test sets.