DLI "Building Transformer-Based Natural Language Processing Applications" Base Environment Container (NVIDIA NeMo)

This container is used in the NVIDIA Deep Learning Institute workshop Building Transformer-Based Natural Language Processing Applications NVIDIA NeMo content, and with it, you can build your own software using the same libraries and tools used in the workshop. If you have not done so yet, we highly recommend you take the course, and check out other self-paced online courses and instructor-led workshops available from the NVIDIA Deep Learning Institute.

Please Note: This container does not include the training materials or data from the workshop, just the base environment.

Prerequisites

The following are required to run this container. For convenience, we also provide, for each, what the DLI used in our workshop.

• NVIDIA Tesla™ GPU architecture or better (we used 1 NVIDIA V100 GPU)
• CUDA 10.2 or later with compatible NVIDIA driver (we used CUDA version 10.2 and driver version 440.33)
• Ubuntu 18.04/20.04 or CentOS 7 (we used Ubuntu 18.04)
• Docker CE v18+ (we used Docker version 18.03.1-ce)
• nvidia-docker v2+ (we used nvidia-docker2)

How to Use the Container

If you've never used Docker, we recommend their Orientation and Setup.

Copy and paste the following to download and run the container:

docker run --runtime=nvidia -d -v $PWD:/dli/task --shm-size=16g -p 8888:8888 -p 6006:6006 --ulimit memlock=-1 --ulimit stack=67108864 nvcr.io/nvidia/dli/dli-nlp-nemo:

After running the above, you will be able to access the Jupyter Lab environment running inside the container by visiting :8888 in your browser. There you will find a notebook, Next_Steps.ipynb containing instructions for a more advanced and open ended project, intended for students who have completed the workshop.

Technical Support

• NVIDIA NeMo Developer Guide v1.0

License

Copyright 2020 NVIDIA

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

Base Image Used for This Container

Also used in this container, and with its own licensing:

• NeMo Container on NGC

Software Installed on Top of Base Image

Also used in this container, and with its own licensing (tag v3-nemo0.11 only):

• BioBERT Base Cased for NeMo Model on NGC
• BioMegatron 345m Base Uncased Model on NGC