NGC | Catalog
CatalogResourcesTAO Toolkit Getting Started

TAO Toolkit Getting Started

For downloads and more information, please view on a desktop device.
Logo for TAO Toolkit Getting Started

Description

Quick start guide for TAO Toolkit.

Publisher

NVIDIA

Use Case

Other

Framework

TransferLearningToolkit

Latest Version

-

Modified

June 14, 2022

Compressed Size

0 B

TAO Toolkit Quick Start

The TAO Launcher is a lightweight Python based CLI application to run TAO. The launcher basically acts as a front-end for the multiple TAO Toolkit containers built on both PyTorch and Tensorflow. The multiple containers essentially get launched automatically based on the type of pretrained model you plan to use for your computer vision or conversational AI use-cases.


TAO Toolkit

The instructions provided in this Getting started guide are for the TAO launcher. If you want to run directly from the containers, refer to our documentation page and quick start video.

Requirements

Minimum Hardware requirements:

The following system configuration is recommended to achieve reasonable training performance with TAO Toolkit and supported models provided:

  • 32 GB system RAM
  • 32 GB of GPU RAM
  • 8 core CPU
  • 1 NVIDIA GPU
  • 100 GB of SSD space

TAO Toolkit is supported on discrete GPUs, such as A100, A40, A30, A2, A16, A100x, A30x, V100, T4, Titan-RTX and Quadro-RTX.

Note: TAO Toolkit is not supported on GPU's before the Pascal generation

Software requirements

Software Version
Ubuntu LTS 20.04
python >=3.6.9<3.7
docker-ce >19.03.5
docker-API 1.40
nvidia-container-toolkit >1.3.0-1
nvidia-container-runtime 3.4.0-1
nvidia-docker2 2.5.0-1
nvidia-driver >510
python-pip >21.06

Installing TAO Toolkit

Installing Pre-requisites

  1. Install docker-ce by following the official instructions

    Once you have installed docker-ce, follow the post-installation steps to ensure that the docker can be run without sudo.

  2. Install nvidia-container-toolkit by following the install-guide).

  3. Get an NGC account and API key:

    • Go to NGC and click the TAO Toolkit container in the Catalog tab. This message is displayed: "Sign in to access the PULL feature of this repository".
    • Enter your Email address and click Next, or click Create an Account.
    • Choose your organization when prompted for Organization/Team.
    • Click Sign In.

  4. Log in to the NGC docker registry (nvcr.io) using the command docker login nvcr.io and enter the following credentials:

    a. Username: "$oauthtoken"
b. Password: "YOUR_NGC_API_KEY"

    where YOUR_NGC_API_KEY corresponds to the key you generated from step 3.

  5. Install miniconda to set-up an isolated python environment

    • Follow the instructions in this link to set up a conda environemnt using miniconda.

    • Once you have installed miniconda, create a new environment by setting the Python version to 3.6.

      conda create -n launcher python=3.6

    • Activate the conda environment that you have just created.

      conda activate launcher

    • Once you have activated your conda environement, the command prompt should show the name of your conda environment.

      (launcher) py-3.6.9 desktop:

    • When you are done with you session, you may deactivate your conda environemnt using the deactivate command:

      conda deactivate

    • You may re-instantiate this created conda environemnt using the following command.

      conda activate launcher

  6. Install Jupyter Lab

    pip3 install jupyter

Download and run sample notebooks

  1. Download Jupyter Notebook and training spec files:

    • Computer Vision

      wget --content-disposition https://api.ngc.nvidia.com/v2/resources/nvidia/tao/cv_samples/versions/v1.4.0/zip -O cv_samples_v1.4.0.zip
unzip -u cv_samples_v1.4.0.zip  -d ./cv_samples_v1.4.0 && rm -rf cv_samples_v1.4.0.zip && cd ./cv_samples_v1.4.0

      This package contains all the notebooks and training spec files to train all CV models in TAO. To select what notebook to use for each model, refer to the table below.

      Purpose-built Model Jupyter notebook
      PeopleNet detectnet_v2/detectnet_v2.ipynb
      TrafficCamNet detectnet_v2/detectnet_v2.ipynb
      DashCamNet detectnet_v2/detectnet_v2.ipynb
      FaceDetectIR detectnet_v2/detectnet_v2.ipynb
      VehicleMakeNet classification/classification.ipynb
      VehicleTypeNet classification/classification.ipynb
      PeopleSegNet mask_rcnn/mask_rcnn.ipynb
      PeopleSemSegNet unet/unet_isbi.ipynb
      Bodypose Estimation bpnet/bpnet.ipynb
      License Plate Detection detectnet_v2/detectnet_v2.ipynb
      License Plate Recognition lprnet/lprnet.ipynb
      Gaze Estimation gazenet/gazenet.ipynb
      Facial Landmark fpenet/fpenet.ipynb
      Heart Rate Estimation heartratenet/heartratenet.ipynb
      Gesture Recognition gesturenet/gesturenet.ipynb
      Emotion Recognition emotionnet/emotionnet.ipynb
      FaceDetect facenet/facenet.ipynb
      ActionRecognitionNet action_recognition_net/actionrecognitionnet.ipynb
      PoseClassificationNet pose_classification_net/pose_classificationnet.ipynb
      Pointpillars pointpillars/pointpillars.ipynb
      Open model architecture Jupyter notebook
      DetectNet_v2 detectnet_v2/detectnet_v2.ipynb
      FasterRCNN faster_rcnn/faster_rcnn.ipynb
      YOLOV3 yolo_v3/yolo_v3.ipynb
      YOLOV4 yolo_v4/yolo_v4.ipynb
      YOLOv4-Tiny yolo_v4_tiny/yolo_v4_tiny.ipynb
      SSD ssd/ssd.ipynb
      DSSD dssd/dssd.ipynb
      RetinaNet retinanet/retinanet.ipynb
      MaskRCNN mask_rcnn/mask_rcnn.ipynb
      UNET unet/unet_isbi.ipynb
      Image Classification classification/classification.ipynb
      EfficientDet efficientdet/efficientdet.ipynb

    • Conversational AI

      For Conversational AI, all notebooks are available on NGC. Please download the notebook from the appropriate NGC resource as mentioned in the table below.

      Conversational AI Task Jupyter Notebooks
      Speech to Text Citrinet Speech to Text Citrinet Notebook
      Speech to Text Conformer Speech to Text Conformer Notebook
      Question Answering Question Answering Notebook
      Text Classification Text Classification Notebook
      Token Classification Token Classification Notebook
      Punctuation and Capitalization Punctuation Capitalization Notebook
      Intent and Slot Classification Intent Slot Classification Notebook
      NGram Language Model NGram Language Model Notebook
      Text to Speech Text to Speech Notebook

  2. Start Jupyter notebook

    jupyter notebook --ip 0.0.0.0 --port 8888 --allow-root

  3. Open an internet browser on localhost and navigate to the following URL: http://0.0.0.0:8888

  4. Open the notebook that you are interested in training and start training.

Note: All the instructions to train, prune, optimize and download pretrained models are provided in the notebook.

Blogs

Synthetic Data and TAO

Action Recognition Blog

Real-time License Plate Detection

2 Pose Estimation: Part 1 Part 2

Building ConvAI with TAO Toolkit

License

TAO Toolkit getting Started License for TAO containers is included within the container at workspace/EULA.pdf. License for the pre-trained models are available with the model files. By pulling and using the Train Adapt Optimize (TAO) Toolkit container to download models, you accept the terms and conditions of these licenses.