Model Overview

This is a language model trained for Automatic Speech Recognition. It can be used as a neural rescorer which can rescore the top candidates predicted by an acoustic model.

Model Architecture

The model is a large Transformer [1] trained for language modelling.

Training

NeMo toolkit [2] was used for training this model. You may find more info on how to train and use neural language models for rescoring of ASR models here: ASR Language Modeling

Datasets

This collection contains the models trained on the text transcripts of the LibriSpeech train set plus the LibriSpeech language modeling text corpus [3].

Performance

This model has been evaluated on some of the ASR models. You may find the performance of this model on some of the ASR models here.

How to Use this Model

The model is available for use in the NeMo toolkit [3], and can be used as a pre-trained checkpoint for inference or for fine-tuning on another dataset.

Automatically load the model from NGC

import nemo.collections.nlp as nemo_nlp
asr_lm_model = nemo_nlp.models.language_modeling.TransformerLMModel.from_pretrained(model_name="asrlm_en_transformer_large_ls")

Evaluate an ASR model with neural rescorer

You may use this script to evaluate this LM model on an ASR model.

Input

This model accepts text as input.

Output

This model estimates the likelihood of each token conditioned on previous tokens.

Limitations

Since this model is trained on LibriSpeech text corpus, the performance of this model might may degrade for other domains.

References

[1] Attention Is All You Need

[2] NVIDIA NeMo Toolkit

[3] LibriSpeech Dataset

Licence

License to use this model is covered by the NGC TERMS OF USE unless another License/Terms Of Use/EULA is clearly specified. By downloading the public and release version of the model, you accept the terms and conditions of the NGC TERMS OF USE.