Repository files navigation

PyTorch implementation ofNatural TTS Synthesis By ConditioningWavenet On Mel Spectrogram Predictions.

This implementation includesdistributed andautomatic mixed precision supportand uses theLJSpeech dataset.

Distributed and Automatic Mixed Precision support relies on NVIDIA'sApex andAMP.

Visit ourwebsite for audio samples using our publishedTacotron 2 andWaveGlow models.

1. NVIDIA GPU + CUDA cuDNN

1. Download and extract theLJ Speech dataset
2. Clone this repo:git clone https://github.com/NVIDIA/tacotron2.git
3. CD into this repo:cd tacotron2
4. Initialize submodule:git submodule init; git submodule update
5. Update .wav paths:sed -i -- 's,DUMMY,ljs_dataset_folder/wavs,g' filelists/*.txt
   • Alternatively, setload_mel_from_disk=True inhparams.py and update mel-spectrogram paths
6. InstallPyTorch 1.0
7. InstallApex
8. Install python requirements or build docker image
   • Install python requirements:pip install -r requirements.txt

1. python train.py --output_directory=outdir --log_directory=logdir
2. (OPTIONAL)tensorboard --logdir=outdir/logdir

Training using a pre-trained model can lead to faster convergence
By default, the dataset dependent text embedding layers areignored

1. Download our publishedTacotron 2 model
2. python train.py --output_directory=outdir --log_directory=logdir -c tacotron2_statedict.pt --warm_start

1. python -m multiproc train.py --output_directory=outdir --log_directory=logdir --hparams=distributed_run=True,fp16_run=True

1. Download our publishedTacotron 2 model
2. Download our publishedWaveGlow model
3. jupyter notebook --ip=127.0.0.1 --port=31337
4. Load inference.ipynb

N.b. When performing Mel-Spectrogram to Audio synthesis, make sure Tacotron 2and the Mel decoder were trained on the same mel-spectrogram representation.

WaveGlow Faster than real time Flow-basedGenerative Network for Speech Synthesis

nv-wavenet Faster than real timeWaveNet.

This implementation uses code from the following repos:KeithIto,PremSeetharaman as described in our code.

We are inspired byRyuchi Yamamoto'sTacotron PyTorch implementation.

We are thankful to the Tacotron 2 paper authors, specially Jonathan Shen, YuxuanWang and Zongheng Yang.