NGC | Catalog
CatalogModelsASR Language Modeling Transformer Large LibriSpeech

ASR Language Modeling Transformer Large LibriSpeech

Logo for ASR Language Modeling Transformer Large LibriSpeech
Transformer-Large language model for English ASR, Trained on LibriSpeech text corpus with NeMo
Latest Version
April 4, 2023
727.75 MB

Model Overview

This is a language model trained for Automatic Speech Recognition. It can be used as a neural rescorer which can rescore the top candidates predicted by an acoustic model.

Model Architecture

The model is a large Transformer [1] trained for language modelling.


NeMo toolkit [2] was used for training this model. You may find more info on how to train and use neural language models for rescoring of ASR models here: ASR Language Modeling


This collection contains the models trained on the text transcripts of the LibriSpeech train set plus the LibriSpeech language modeling text corpus [3].


This model has been evaluated on some of the ASR models. You may find the performance of this model on some of the ASR models here.

How to Use this Model

The model is available for use in the NeMo toolkit [3], and can be used as a pre-trained checkpoint for inference or for fine-tuning on another dataset.

Automatically load the model from NGC

import nemo.collections.nlp as nemo_nlp
asr_lm_model = nemo_nlp.models.language_modeling.TransformerLMModel.from_pretrained(model_name="asrlm_en_transformer_large_ls")

Evaluate an ASR model with neural rescorer

You may use this script to evaluate this LM model on an ASR model.


This model accepts text as input.


This model estimates the likelihood of each token conditioned on previous tokens.


Since this model is trained on LibriSpeech text corpus, the performance of this model might may degrade for other domains.


[1] Attention Is All You Need

[2] NVIDIA NeMo Toolkit

[3] LibriSpeech Dataset


License to use this model is covered by the NGC TERMS OF USE unless another License/Terms Of Use/EULA is clearly specified. By downloading the public and release version of the model, you accept the terms and conditions of the NGC TERMS OF USE.