TAO Toolkit Computer Vision Sample Workflows

TAO Toolkit Computer Vision Sample Workflows

Logo for TAO Toolkit Computer Vision Sample Workflows
Sample notebooks for TAO Toolkit Computer Vision workflows.
Latest Version
April 4, 2023
Compressed Size
2.96 MB

TAO CV Sample Workflows

Train Adapt Optimize (TAO) Toolkit is a python based AI toolkit for taking purpose-built pre-trained AI models and customizing them with your own data. TAO adapts popular network architectures and backbones to your data, allowing you to train, fine tune, prune and export highly optimized and accurate AI models for edge deployment.

The pre-trained models accelerate the AI training process and reduce costs associated with large scale data collection, labeling, and training models from scratch. Transfer learning with pre-trained models can be used for AI applications in smart cities, retail, healthcare, industrial inspection and more.

Build end-to-end services and solutions for transforming pixels and sensor data to actionable insights using TAO, DeepStream SDK and TensorRT. TAO can train models for common vision AI tasks such as object detection, classification, instance segmentation as well as other complex tasks such as facial landmark, gaze estimation, heart rate estimation and others.

This resource lists out several sample notebooks to walk you through full training workflow using TAO 3.0.

Getting Started

To get started, first choose the model architecture that you want to build, then select the appropriate model card on NGC and then choose one of the supported backbones.


Running TAO Toolkit

  1. Setup your python environment using python virtualenv and virtualenvwrapper.

  2. In TAO Toolkit, we have created an abstraction above the container, you will launch all your training jobs from the launcher. No need to manually pull the appropriate container, tao-launcher will handle that. You may install the launcher using pip with the following commands.

pip3 install nvidia-pyindex
pip3 install nvidia-tao
  1. Download the sample jupyter notebooks using the command mentioned in the CLI tab and kick start a jupyter server using the following command
jupyter notebook --ip --port 8888 --allow-root
Purpose-built Model Jupyter notebook
PeopleNet detectnet_v2/detectnet_v2.ipynb
TrafficCamNet detectnet_v2/detectnet_v2.ipynb
DashCamNet detectnet_v2/detectnet_v2.ipynb
FaceDetectIR detectnet_v2/detectnet_v2.ipynb
VehicleMakeNet classification/classification.ipynb
VehicleTypeNet classification/classification.ipynb
PeopleSegNet mask_rcnn/mask_rcnn.ipynb
PeopleSemSegNet unet/unet_isbi.ipynb
Bodypose Estimation bpnet/bpnet.ipynb
License Plate Detection detectnet_v2/detectnet_v2.ipynb
License Plate Recognition lprnet/lprnet.ipynb
Gaze Estimation gazenet/gazenet.ipynb
Facial Landmark fpenet/fpenet.ipynb
Heart Rate Estimation heartratenet/heartratenet.ipynb
Gesture Recognition gesturenet/gesturenet.ipynb
Emotion Recognition emotionnet/emotionnet.ipynb
FaceDetect facenet/facenet.ipynb
Open model architecture Jupyter notebook
DetectNet_v2 detectnet_v2/detectnet_v2.ipynb
FasterRCNN faster_rcnn/faster_rcnn.ipynb
YOLOV3 yolo_v3/yolo_v3.ipynb
YOLOV4 yolo_v4/yolo_v4.ipynb
SSD ssd/ssd.ipynb
DSSD dssd/dssd.ipynb
RetinaNet retinanet/retinanet.ipynb
MaskRCNN mask_rcnn/mask_rcnn.ipynb
UNET unet/unet_isbi.ipynb
Image Classification classification/classification.ipynb