NeMo - Natural Language Processing

Overview

NVIDIA NeMo toolkit supports Natural Language Processing (NLP) models for the following tasks:

• Language modelling (BERT, Megatron, etc)
• Extractive questions answering
• Named entity recognition
• Text classification
• Intent prediction and slot filling
• Automatic text punctuation and capitalization
• Neural machine translation

For detailed information regarding NeMo's NLP capabilities, visit the NeMo NLP documentation page.

Usage

You can instantiate many pretrained models automatically directly from NGC. To do so, start your script with:

import nemo
import nemo.collections.nlp as nemo_nlp

Then chose what type of model you would like to instantiate. See table below for the list of models that are available for each task. For example:

# Neural Machine Translation model for Russian to English translation
nmt_model = nemo_nlp.models.MTEncDecModel.from_pretrained(model_name='nmt_ru_en_transformer6x6').cuda()
# Use it to translate Russian text to English
english_text = nmt_model.translate(["Привет мир!"])
# This will print: "Hello world!"
print(english_text)

Note that you can also list all available models using API by calling <base_class>.list_available_models(...) method. You can also download a models ".nemo" files from the "File Browser" tab and then instantiate those models with <base_class>.restore_from(PATH_TO_DOTNEMO_FILE) method. In this case, make sure you are matching NeMo and models' versions.

Available pre-trained models

Model Name	Task	Model Card
punctuation_en_bert	Punctuation & Capitalization	NGC Model Card
punctuation_en_distilbert	Punctuation & Capitalization	NGC Model Card
ner_en_bert	Named Entity Recognition	NGC Model Card
bertbaseuncased	Language Modeling	NGC Model Card
bertlargeuncased	Language Modeling	NGC Model Card
qa_squadv1_1_bertbase	Question Answering	NGC Model Card
qa_squadv2_0_bertbase	Question Answering	NGC Model Card
qa_squadv1_1_bertlarge	Question Answering	NGC Model Card
qa_squadv2_0_bertlarge	Question Answering	NGC Model Card
qa_squadv1_1_megatron_cased	Question Answering	NGC Model Card
qa_squadv2_0_megatron_cased	Question Answering	NGC Model Card
qa_squadv1_1_megatron_uncased	Question Answering	NGC Model Card
qa_squadv2_0_megatron_uncased	Question Answering	NGC Model Card
nmt_en_de_transformer12x2	Machine Translation	NGC Model Card
nmt_de_en_transformer12x2	Machine Translation	NGC Model Card
nmt_en_es_transformer12x2	Machine Translation	NGC Model Card
nmt_es_en_transformer12x2	Machine Translation	NGC Model Card
nmt_en_fr_transformer12x2	Machine Translation	NGC Model Card
nmt_fr_en_transformer12x2	Machine Translation	NGC Model Card
nmt_en_ru_transformer6x6	Machine Translation	NGC Model Card
nmt_ru_en_transformer6x6	Machine Translation	NGC Model Card
nmt_zh_en_transformer6x6	Machine Translation	NGC Model Card
nmt_en_zh_transformer6x6	Machine Translation	NGC Model Card

Compatibility with HuggingFace and Megatron language models.

It is possible to use pre-trained models from HuggingFace transformers library and NVIDIA Megatron as encoders for various NeMo NLP models. Please refer to this documentation section for details.