What is NeMo?

NVIDIA NeMo is an open source toolkit for conversational AI. It is built for data scientists and researchers to build new state of the art speech and NLP networks easily through API compatible building blocks that can be connected together.

Neural Modules are conceptual blocks that take typed inputs and produce typed outputs. Such modules represent data layers, encoders, decoders, language models, loss functions, or methods of combining activations. NeMo makes it easy to combine and re-use these building blocks while providing a level of semantic correctness checking via its neural type system.

Conversational AI architectures are typically very large and require a lot of data and compute for training. Built for speed, NeMo can utilize NVIDIA's Tensor Cores and scale out training to multiple GPUs and multiple nodes.NeMo uses PyTorch Lightning for easy and performant multi-GPU/multi-node mixed-precision training. Every NeMo model is a LightningModule that comes equipped with all supporting infrastructure for training and reproducibility. Conversational AI architectures are typically very large and require a lot of data and compute for training. Built for speed, NeMo can utilize NVIDIA's Tensor Cores and scale out training to multiple GPUs and multiple nodes.NeMo uses PyTorch Lightning for easy and performant multi-GPU/multi-node mixed-precision training. Every NeMo model is a LightningModule that comes equipped with all supporting infrastructure for training and reproducibility.

Several pretrained models for Automatic Speech Recognition(ASR), Natural Language Processing(NLP) and Text to Speech(TTS) are provided in NGC Collection for NeMo

What's new in Release 1.0beta or 1.0.0b1 ?

This release updates core training api with Pytorch Lightning. Every NeMo model is a LightningModule that comes equipped with all supporting infrastructure for training and reproducibility. Every NeMo model has an example configuration file and a corresponding script that contains all configurations needed for training.

NeMo, Pytorch Lightning, and Hydra makes all NeMo models have the same look and feel so that it is easy to do Conversational AI research across multiple domains.

New models such as Speaker Identification and Megatron BERT provide variety. Together with the collection and docker container, we believe NeMo is on track to become a premier toolkit for Conversational AI model building and training.

Github link: https://github.com/NVIDIA/NeMo

Installation and Getting Started

Pull the docker: docker pull nvcr.io/nvidia/nemo:v1.0.0b1

Run: docker run --runtime=nvidia -it --rm -v --shm-size=16g -p 8888:8888 -p 6006:6006 --ulimit memlock=-1 --ulimit stack=67108864 nvcr.io/nvidia/nemo:v1.0.0b1

Downloading pretrained weights for fine tuning

Several pretrained models in the form of Pytorch checkpoints are provided with the NeMo toolkit. Complete list of models is available in the NGC collection

Models trained with NeMo are high accuracy and trained on muliple datasets. Use the docker container to get started and check out all the models in the collection

Documentation

NeMo developer guide is available here

Known Issues

• The PyTorch container includes a version of Django with a known vulnerability that was discovered late in our QA process. See CVE-2021-31542 for details. This will be fixed in the next release.

• The PyTorch container includes a version of Pillow with known vulnerabilities discovered late in our QA process. See CVE-2021-25287, CVE-2021-28676, CVE-2021-28677, and CVE-2021-25288 for details. This will be fixed in the next release.

License

NeMo is licensed under Apache License 2.0 Link Here. By pulling and using the container, you accept the terms and conditions of this license.

Technical Support

Use the Github Issues forum for questions regarding this Software