Towards Fine-Grained Prosody Control for Voice Conversion

In a typical voice conversion system, previous works utilized various acoustic features (such as the pitch, voiced/unvoiced flag and aperiodicity) of the source speech to control the prosody of converted speech. However, prosody is related with many factors, such as the intonation, stress and rhythm. It is a challenging task to perfectly describe prosody through handcrafted acoustic features. To address these difficulties, we propose to use prosody embeddings to describe prosody. These embeddings are learned from the source speech in an unsupervised manner. To verify the effectiveness of our proposed method, we conduct experiments on our Mandarin corpus. Experimental results show that our proposed method can improve the speech quality and speaker similarity of the converted speech. What's more, we observe that our method can even achieve promising results in singing conditions.