Overview

NVIDIA NeMo toolkit supports Natural Language Processing (NLP) models for the following tasks:

• Language modelling (BERT, Megatron, etc)
• Extractive questions answering
• Named entity recognition
• Text classification
• Intent prediction and slot filling
• Automatic text punctuation and capitalization
• Neural machine translation

For detailed information regarding NeMo's NLP capabilities, visit the NeMo NLP documentation page.

Usage

You can instantiate many pretrained models automatically directly from NGC. To do so, start your script with:

import nemo
import nemo.collections.nlp as nemo_nlp

Then chose what type of model you would like to instantiate. See table below for the list of models that are available for each task. For example:

# Neural Machine Translation model for Russian to English translation
nmt_model = nemo_nlp.models.MTEncDecModel.from_pretrained(model_name='nmt_ru_en_transformer6x6').cuda()
# Use it to translate Russian text to English
english_text = nmt_model.translate(["Привет мир!"])
# This will print: "Hello world!"
print(english_text)

Note that you can also list all available models using API by calling <base_class>.list_available_models(...) method. You can also download a models ".nemo" files from the "File Browser" tab and then instantiate those models with <base_class>.restore_from(PATH_TO_DOTNEMO_FILE) method. In this case, make sure you are matching NeMo and models' versions.

Available pre-trained models

Compatibility with HuggingFace and Megatron language models.

It is possible to use pre-trained models from HuggingFace transformers library and NVIDIA Megatron as encoders for various NeMo NLP models. Please refer to this documentation section for details.