NGC | Catalog
CatalogModelsSpeech Synthesis Waveglow

Speech Synthesis Waveglow

For downloads and more information, please view on a desktop device.
Logo for Speech Synthesis Waveglow

Description

Universal waveform generator from mel-spectrograms.

Publisher

NVIDIA

Use Case

Text To Speech

Framework

Transfer Learning Toolkit

Latest Version

deployable_v1.0

Modified

April 8, 2022

Size

326.36 MB

Speech Synthesis: WaveGlow Model Card =====================================

Model overview --------------

WaveGlow is a neural vocoder model for text-to-speech applications. It is intended as the second part of a two-stage speech synthesis pipeline, with a mel-spectrogram generator such as Tacotron2 as the first stage.

Model architecture ------------------

WaveGlow is a Glow-based (alternatively flow-based) model that generates audio conditioned on mel spectrograms. WaveGlow is a reversible neural network. It can be run in two modes: the first mode takes audio and transforms it to samples drawn from a normal distribution, and the second mode takes samples from a normal distribution and transforms it to audio. Both modes are conditioned on a mel spectrogram.

Training --------

This model can be used to generate most voices in most languages without retraining. We have observed this trained WaveGlow to generate English audio and Mandarin audio.

Dataset

This model is trained on the LJSpeech dataset sampled at 22050 Hz.

How to use this model ---------------------

WaveGlow is intended to be used as the second part of a two stage speech synthesis pipeline. WaveGlow takes a mel spectrogram and returns audio.

Input

Mel spectrogram of shape (batch x mel_channels x time)

Output

Audio of shape (batch x time)

Limitations -----------

N/A

References ----------

WaveGlow paper: https://arxiv.org/abs/1811.00002

License -------

By downloading and using the models and resources packaged with TLT Conversational AI, you would be accepting the terms of the Riva license

Ethical AI ----------

NVIDIA’s platforms and application frameworks enable developers to build a wide array of AI applications. Consider potential algorithmic bias when choosing or creating the models being deployed. Work with the model’s developer to ensure that it meets the requirements for the relevant industry and use case; that the necessary instruction and documentation are provided to understand error rates, confidence intervals, and results; and that the model is being used under the conditions and in the manner intended.