Speech Recognition: Conformer

Model Overview

Conformer-CTC (around 120M parameters) is trained on ASRSet with over 3500 hours of German (de-DE) speech. The model transcribes speech in lower case German alphabet along with spaces and apostrophes.

Model Architecture

Conformer-CTC model is a non-autoregressive variant of Conformer model for Automatic Speech Recognition which uses CTC loss/decoding instead of Transducer. For more information, refer to the Conformer-CTC Model documentation.

Intended Use

Primary use case intended for these models is automatic speech recognition.

Input

Single-channel audio files (WAV) with a 16kHz sample rate

Output

Transcripts, which are sequences of valid vocabulary labels as given by the specification file

How to Use this Model

Conformer is an end-to-end architecture that is trained using CTC loss. These model checkpoints are intended to be used with NVIDIA Riva.

The models are encrypted with the key tlt_encode.

Training

The model was trained on various proprietary and open-source datasets. These datasets include domain specific data for various domains, spontaneous speech and dialog, all of which contribute to the model’s accuracy.

This model delivers WER that is better than or comparable to popular alternate Speech to Text solutions for a range of domains and use cases.

Limitations

Currently, Conformer models only support training and inference on .wav audio files. All models included here were trained and evaluated on audio files with a sample rate of 16kHz, so for best performance you may need to upsample or downsample audio files to 16kHz.

In addition, the model will perform best on audio samples that are longer than 0.1 seconds long. For training and fine-tuning Conformer models, it is recommended that samples are capped at a maximum length of around 15 seconds, depending on the amount of memory available to you. You do not need to place a maximum length limitation for evaluation.

Suggested Reading

Refer to the Riva documentation for more information.

License

By downloading and using the models and resources packaged with Riva Conversational AI, you accept the terms of the Riva license.

Ethical AI

NVIDIA’s platforms and application frameworks enable developers to build a wide array of AI applications. Consider potential algorithmic bias when choosing or creating the models being deployed. Work with the model’s developer to ensure that it meets the requirements for the relevant industry and use case; that the necessary instruction and documentation are provided to understand error rates, confidence intervals, and results; and that the model is being used under the conditions and in the manner intended.