TAO Toolkit Quick Start

The TAO Launcher is a lightweight Python based CLI application to run TAO. The launcher basically acts as a front-end for the multiple TAO Toolkit containers built on both PyTorch and Tensorflow. The multiple containers essentially get launched automatically based on the type of pretrained model you plan to use for your computer vision or conversational AI use-cases.

TAO Toolkit

The instructions provided in this Getting started guide are for the TAO launcher. If you want to run directly from the containers, refer to our documentation page and quick start video.

Requirements

Minimum Hardware requirements:

The following system configuration is recommended to achieve reasonable training performance with TAO Toolkit and supported models provided:

• 32 GB system RAM
• 32 GB of GPU RAM
• 8 core CPU
• 1 NVIDIA GPU
• 100 GB of SSD space

TAO Toolkit is supported on discrete GPUs, such as A100, A40, A30, A2, A16, A100x, A30x, V100, T4, Titan-RTX and Quadro-RTX.

Note: TAO Toolkit is not supported on GPU's before the Pascal generation

Software requirements

Software	Version
Ubuntu LTS	20.04
python	>=3.6.9<3.7
docker-ce	>19.03.5
docker-API	1.40
nvidia-container-toolkit	>1.3.0-1
nvidia-container-runtime	3.4.0-1
nvidia-docker2	2.5.0-1
nvidia-driver	>510
python-pip	>21.06

Installing TAO Toolkit

Installing Pre-requisites

1. Install docker-ce by following the official instructions

   Once you have installed docker-ce, follow the post-installation steps to ensure that the docker can be run without sudo.

2. Install nvidia-container-toolkit by following the install-guide).

3. Get an NGC account and API key:

   • Go to NGC and click the TAO Toolkit container in the Catalog tab. This message is displayed: "Sign in to access the PULL feature of this repository".
   • Enter your Email address and click Next, or click Create an Account.
   • Choose your organization when prompted for Organization/Team.
   • Click Sign In.

4. Log in to the NGC docker registry (nvcr.io) using the command docker login nvcr.io and enter the following credentials:

   a. Username: "$oauthtoken"
   b. Password: "YOUR_NGC_API_KEY"
   

   where YOUR_NGC_API_KEY corresponds to the key you generated from step 3.

5. Install miniconda to set-up an isolated python environment

   • Follow the instructions in this link to set up a conda environemnt using miniconda.

   • Once you have installed miniconda, create a new environment by setting the Python version to 3.6.

     conda create -n launcher python=3.6
     

   • Activate the conda environment that you have just created.

     conda activate launcher
     

   • Once you have activated your conda environement, the command prompt should show the name of your conda environment.

     (launcher) py-3.6.9 desktop:
     

   • When you are done with you session, you may deactivate your conda environemnt using the deactivate command:

     conda deactivate
     

   • You may re-instantiate this created conda environemnt using the following command.

     conda activate launcher
     

6. Install Jupyter Lab

   pip3 install jupyter
   

Download and run sample notebooks

1. Download Jupyter Notebook and training spec files:

   • Computer Vision

     wget --content-disposition https://api.ngc.nvidia.com/v2/resources/nvidia/tao/cv_samples/versions/v1.4.0/zip -O cv_samples_v1.4.0.zip
     unzip -u cv_samples_v1.4.0.zip  -d ./cv_samples_v1.4.0 && rm -rf cv_samples_v1.4.0.zip && cd ./cv_samples_v1.4.0
     

     This package contains all the notebooks and training spec files to train all CV models in TAO. To select what notebook to use for each model, refer to the table below.

     Purpose-built Model	Jupyter notebook
     PeopleNet	detectnet_v2/detectnet_v2.ipynb
     TrafficCamNet	detectnet_v2/detectnet_v2.ipynb
     DashCamNet	detectnet_v2/detectnet_v2.ipynb
     FaceDetectIR	detectnet_v2/detectnet_v2.ipynb
     VehicleMakeNet	classification/classification.ipynb
     VehicleTypeNet	classification/classification.ipynb
     PeopleSegNet	mask_rcnn/mask_rcnn.ipynb
     PeopleSemSegNet	unet/unet_isbi.ipynb
     Bodypose Estimation	bpnet/bpnet.ipynb
     License Plate Detection	detectnet_v2/detectnet_v2.ipynb
     License Plate Recognition	lprnet/lprnet.ipynb
     Gaze Estimation	gazenet/gazenet.ipynb
     Facial Landmark	fpenet/fpenet.ipynb
     Heart Rate Estimation	heartratenet/heartratenet.ipynb
     Gesture Recognition	gesturenet/gesturenet.ipynb
     Emotion Recognition	emotionnet/emotionnet.ipynb
     FaceDetect	facenet/facenet.ipynb
     ActionRecognitionNet	action_recognition_net/actionrecognitionnet.ipynb
     PoseClassificationNet	pose_classification_net/pose_classificationnet.ipynb
     Pointpillars	pointpillars/pointpillars.ipynb

     Open model architecture	Jupyter notebook
     DetectNet_v2	detectnet_v2/detectnet_v2.ipynb
     FasterRCNN	faster_rcnn/faster_rcnn.ipynb
     YOLOV3	yolo_v3/yolo_v3.ipynb
     YOLOV4	yolo_v4/yolo_v4.ipynb
     YOLOv4-Tiny	yolo_v4_tiny/yolo_v4_tiny.ipynb
     SSD	ssd/ssd.ipynb
     DSSD	dssd/dssd.ipynb
     RetinaNet	retinanet/retinanet.ipynb
     MaskRCNN	mask_rcnn/mask_rcnn.ipynb
     UNET	unet/unet_isbi.ipynb
     Image Classification	classification/classification.ipynb
     EfficientDet	efficientdet/efficientdet.ipynb

   • Conversational AI

     For Conversational AI, all notebooks are available on NGC. Please download the notebook from the appropriate NGC resource as mentioned in the table below.

     Conversational AI Task	Jupyter Notebooks
     Speech to Text Citrinet	Speech to Text Citrinet Notebook
     Speech to Text Conformer	Speech to Text Conformer Notebook
     Question Answering	Question Answering Notebook
     Text Classification	Text Classification Notebook
     Token Classification	Token Classification Notebook
     Punctuation and Capitalization	Punctuation Capitalization Notebook
     Intent and Slot Classification	Intent Slot Classification Notebook
     NGram Language Model	NGram Language Model Notebook
     Text to Speech	Text to Speech Notebook

2. Start Jupyter notebook

   jupyter notebook --ip 0.0.0.0 --port 8888 --allow-root
   

3. Open an internet browser on localhost and navigate to the following URL: http://0.0.0.0:8888

4. Open the notebook that you are interested in training and start training.

Note: All the instructions to train, prune, optimize and download pretrained models are provided in the notebook.

Important Links

• TAO Toolkit Documentation
• TAO for CV (TensorFlow)
• TAO for CV (PyTorch)
• TAO for Conversational AI

Blogs

Synthetic Data and TAO

Action Recognition Blog

Real-time License Plate Detection

2 Pose Estimation: Part 1 Part 2

Building ConvAI with TAO Toolkit

License

TAO Toolkit getting Started License for TAO containers is included within the container at workspace/EULA.pdf. License for the pre-trained models are available with the model files. By pulling and using the Train Adapt Optimize (TAO) Toolkit container to download models, you accept the terms and conditions of these licenses.