NeMo

What is NeMo?

NVIDIA NeMo is an open source toolkit for conversational AI. It is built for data scientists and researchers to build new state of the art speech and NLP networks easily through API compatible building blocks that can be connected together.

Neural Modules are conceptual blocks that take typed inputs and produce typed outputs. Such modules represent data layers, encoders, decoders, language models, loss functions, or methods of combining activations. NeMo makes it easy to combine and re-use these building blocks while providing a level of semantic correctness checking via its neural type system.

Conversational AI architectures are typically very large and require a lot of data and compute for training. Built for speed, NeMo can utilize NVIDIA's Tensor Cores and scale out training to multiple GPUs and multiple nodes.NeMo uses PyTorch Lightning for easy and performant multi-GPU/multi-node mixed-precision training. Every NeMo model is a LightningModule that comes equipped with all supporting infrastructure for training and reproducibility. Conversational AI architectures are typically very large and require a lot of data and compute for training. Built for speed, NeMo can utilize NVIDIA's Tensor Cores and scale out training to multiple GPUs and multiple nodes.NeMo uses PyTorch Lightning for easy and performant multi-GPU/multi-node mixed-precision training. Every NeMo model is a LightningModule that comes equipped with all supporting infrastructure for training and reproducibility.

Several pretrained models for Automatic Speech Recognition (ASR), Natural Language Processing (NLP) and Text to Speech (TTS) are provided in NGC Collection for NeMo

What's new in NeMo 1.x ?

This release updates core training api with Pytorch Lightning. Every NeMo model is a LightningModule that comes equipped with all supporting infrastructure for training and reproducibility. Every NeMo model has an example configuration file and a corresponding script that contains all configurations needed for training.

NeMo, Pytorch Lightning, and Hydra makes all NeMo models have the same look and feel so that it is easy to do Conversational AI research across multiple domains.

New models such as Speaker Identification and Megatron BERT provide variety. Together with the collection and docker container, we believe NeMo is on track to become a premier toolkit for Conversational AI model building and training.

Github link: https://github.com/NVIDIA/NeMo

Release Notes

Please find the release notes of this NeMo version at https://github.com/NVIDIA/NeMo/releases/tag/v1.21.0

Installation and Getting Started

Pull the docker: docker pull nvcr.io/nvidia/nemo:23.08

Run: docker run --runtime=nvidia -it --rm -v --shm-size=16g -p 8888:8888 -p 6006:6006 --ulimit memlock=-1 --ulimit stack=67108864 nvcr.io/nvidia/nemo:23.08

Downloading pretrained weights for fine tuning

Several pretrained models in the form of Pytorch checkpoints are provided with the NeMo toolkit. Complete list of models is available in the NGC collection

Models trained with NeMo are high accuracy and trained on multiple datasets. Use the docker container to get started and check out all the models in the collection

Documentation

NeMo developer guide is available here

Known Issues

• The PyTorch container (23.08) includes a series of vulnerabilities that were detected with updated scans post the release of the container. As NeMo uses these containers as its base, it too is affected by the vulnerabilities listed here. Besides it also has vulnerability CVE-2023-39332 introduced by node.

License

NeMo is licensed under Apache License 2.0 Link Here. By pulling and using the container, you accept the terms and conditions of this license.

Technical Support

Use the Github Issues forum for questions regarding this Software