The TAO Launcher is a lightweight Python based CLI application to run TAO. The launcher basically acts as a front-end for the multiple TAO Toolkit containers built on both PyTorch and Tensorflow. The multiple containers essentially get launched automatically based on the type of pretrained model you plan to use for your computer vision or conversational AI use-cases.
TAO Toolkit
The instructions provided in this Getting started guide are for the TAO launcher. If you want to run directly from the containers, refer to our documentation page and quick start video.
The following system configuration is recommended to achieve reasonable training performance with TAO Toolkit and supported models provided:
TAO Toolkit is supported on discrete GPUs, such as A100, A40, A30, A2, A16, A100x, A30x, V100, T4, Titan-RTX and Quadro-RTX.
Note: TAO Toolkit is not supported on GPU's before the Pascal generation
Software | Version |
---|---|
Ubuntu LTS | 20.04 |
python | >=3.6.9<3.7 |
docker-ce | >19.03.5 |
docker-API | 1.40 |
nvidia-container-toolkit |
>1.3.0-1 |
nvidia-container-runtime | 3.4.0-1 |
nvidia-docker2 | 2.5.0-1 |
nvidia-driver | >510 |
python-pip | >21.06 |
Install docker-ce
by following the official instructions
Once you have installed docker-ce, follow the post-installation steps to ensure that the docker can be run without sudo
.
Install nvidia-container-toolkit
by following the install-guide).
Get an NGC account and API key:
Log in to the NGC docker registry (nvcr.io
) using the command
docker login nvcr.io
and enter the following credentials:
a. Username: "$oauthtoken"
b. Password: "YOUR_NGC_API_KEY"
where YOUR_NGC_API_KEY
corresponds to the key you generated from step 3.
Install miniconda to set-up an isolated python environment
Follow the instructions in this link to set up a conda environemnt using miniconda.
Once you have installed miniconda
, create a new environment by setting the Python version to 3.6.
conda create -n launcher python=3.6
Activate the conda
environment that you have just created.
conda activate launcher
Once you have activated your conda
environement, the command prompt should show the name of
your conda environment.
(launcher) py-3.6.9 desktop:
When you are done with you session, you may deactivate your conda
environemnt using the
deactivate
command:
conda deactivate
You may re-instantiate this created conda
environemnt using the following command.
conda activate launcher
Install Jupyter Lab
pip3 install jupyter
Download Jupyter Notebook and training spec files:
Computer Vision
wget --content-disposition https://api.ngc.nvidia.com/v2/resources/nvidia/tao/cv_samples/versions/v1.4.0/zip -O cv_samples_v1.4.0.zip
unzip -u cv_samples_v1.4.0.zip -d ./cv_samples_v1.4.0 && rm -rf cv_samples_v1.4.0.zip && cd ./cv_samples_v1.4.0
This package contains all the notebooks and training spec files to train all CV models in TAO. To select what notebook to use for each model, refer to the table below.
Purpose-built Model | Jupyter notebook |
---|---|
PeopleNet | detectnet_v2/detectnet_v2.ipynb |
TrafficCamNet | detectnet_v2/detectnet_v2.ipynb |
DashCamNet | detectnet_v2/detectnet_v2.ipynb |
FaceDetectIR | detectnet_v2/detectnet_v2.ipynb |
VehicleMakeNet | classification/classification.ipynb |
VehicleTypeNet | classification/classification.ipynb |
PeopleSegNet | mask_rcnn/mask_rcnn.ipynb |
PeopleSemSegNet | unet/unet_isbi.ipynb |
Bodypose Estimation | bpnet/bpnet.ipynb |
License Plate Detection | detectnet_v2/detectnet_v2.ipynb |
License Plate Recognition | lprnet/lprnet.ipynb |
Gaze Estimation | gazenet/gazenet.ipynb |
Facial Landmark | fpenet/fpenet.ipynb |
Heart Rate Estimation | heartratenet/heartratenet.ipynb |
Gesture Recognition | gesturenet/gesturenet.ipynb |
Emotion Recognition | emotionnet/emotionnet.ipynb |
FaceDetect | facenet/facenet.ipynb |
ActionRecognitionNet | action_recognition_net/actionrecognitionnet.ipynb |
PoseClassificationNet | pose_classification_net/pose_classificationnet.ipynb |
Pointpillars | pointpillars/pointpillars.ipynb |
Open model architecture | Jupyter notebook |
---|---|
DetectNet_v2 | detectnet_v2/detectnet_v2.ipynb |
FasterRCNN | faster_rcnn/faster_rcnn.ipynb |
YOLOV3 | yolo_v3/yolo_v3.ipynb |
YOLOV4 | yolo_v4/yolo_v4.ipynb |
YOLOv4-Tiny | yolo_v4_tiny/yolo_v4_tiny.ipynb |
SSD | ssd/ssd.ipynb |
DSSD | dssd/dssd.ipynb |
RetinaNet | retinanet/retinanet.ipynb |
MaskRCNN | mask_rcnn/mask_rcnn.ipynb |
UNET | unet/unet_isbi.ipynb |
Image Classification | classification/classification.ipynb |
EfficientDet | efficientdet/efficientdet.ipynb |
Conversational AI
For Conversational AI, all notebooks are available on NGC. Please download the notebook from the appropriate NGC resource as mentioned in the table below.
Conversational AI Task | Jupyter Notebooks |
---|---|
Speech to Text Citrinet | Speech to Text Citrinet Notebook |
Speech to Text Conformer | Speech to Text Conformer Notebook |
Question Answering | Question Answering Notebook |
Text Classification | Text Classification Notebook |
Token Classification | Token Classification Notebook |
Punctuation and Capitalization | Punctuation Capitalization Notebook |
Intent and Slot Classification | Intent Slot Classification Notebook |
NGram Language Model | NGram Language Model Notebook |
Text to Speech | Text to Speech Notebook |
Start Jupyter notebook
jupyter notebook --ip 0.0.0.0 --port 8888 --allow-root
Open an internet browser on localhost and navigate to the following URL: http://0.0.0.0:8888
Open the notebook that you are interested in training and start training.
Note: All the instructions to train, prune, optimize and download pretrained models are provided in the notebook.
Real-time License Plate Detection
2 Pose Estimation: Part 1 Part 2
Building ConvAI with TAO Toolkit
TAO Toolkit getting Started
License for TAO containers is included within the container at workspace/EULA.pdf
. License for the pre-trained models are available with the model files. By pulling and using the Train Adapt Optimize (TAO) Toolkit container to download models, you accept the terms and conditions of these licenses.