HiFi-GAN is a generative adversarial network (GAN) model that generates audio from mel spectrograms. The generator uses transposed convolutions to upsample mel-spectrograms to audio.
This model is trained on LJSpeech sampled at 22050Hz, and has been tested on generating female English voices with an American accent.
No performance information available at this time.
This model can be automatically loaded from NGC.
NOTE: In order to generate audio, you also need a spectrogram generator from NeMo. This example uses the FastPitch model.
# Load PastPitch from nemo.collections.tts.models import FastPitchModel spec_generator = FastPitchModel.from_pretrained("tts_en_fastpitch") # Load HiFi-GAN from nemo.collections.tts.models import HifiGanModel model = HifiGanModel.from_pretrained(model_name="tts_en_lj_hifigan_ft_tacotron2_talknet_fastpitch") # Generate audio import soundfile as sf parsed = spec_generator.parse("You can type your sentence here to get nemo to produce speech.") spectrogram = spec_generator.generate_spectrogram(tokens=parsed) audio = model.convert_spectrogram_to_audio(spec=spectrogram) ### Save the audio to disk in a file called speech.wav sf.write("speech.wav", audio.to('cpu').numpy(), 22050)
This model accepts batches of mel spectrograms.
This model outputs audio at 22050Hz.
There are no known limitations at this time.
1.0.0: Add model (
tts_en_lj_hifigan_ft_tacotron2_talknet_fastpitch.nemo) which was released with NeMo 1.0.0. This model was trained on ground-truth mel-spectrograms and additionally fine-tuned on generated spectrograms from Tacotron 2, TalkNet 2 and FastPitch.
1.6.0: Add models (
tts_en_lj_hifigan_ft_mixerttsx.nemo) which were released with NeMo 1.6.0 and model (
tts_en_lj_hifigan_ft_tacotron2_talknet_fastpitch.nemo) from 1.0.0 without changes. The
tts_en_lj_hifigan_ft_mixerttsx.nemo models were fine-tuned on generated spectrograms from Mixer-TTS and Mixer-TTS-X respectively. The
tts_en_lj_hifigan_ft_tacotron2_talknet_fastpitch.nemo model were used as initial checkpoint for fine-tuning.
HiFi-GAN paper: https://arxiv.org/abs/2010.05646