Voice imitation based on speaker adaptivemulti-speaker speech synthesis model

Abstract

Recently-developed deep learning based text-to-speech (TTS) models showed promising performance and possibility of multi-speaker TTS. However, current multi-speaker TTS models are not easily extensible to new speaker's voice and requires much time to retrain the model with new speaker's voice. Our approach can instantly imitate new speaker's voice using speaker adaptation technique. We proposed novel network architecture to enable this task and generated speech samples that are comparable to the existing multi-speaker TTS model. Moreover, we made improvement on baseline TTS model Tacotron by introducing additional connections. We also proposed and demonstrated a way to train speaker embedding to generate arbitrary voices by tuning its value.