TAO Toolkit
Linux / amd64
What is TAO Toolkit?
TAO (Train Adapt Optimize) is a python based AI toolkit that's built on TensorFlow and PyTorch. It provides transfer learning capability to adapt popular neural network architectures and backbones to your data, allowing you to train, fine-tune, prune, quantize and export highly optimized and accurate AI models for edge deployment.
The purpose built pre-trained models accelerate the AI training process and reduce costs associated with large scale data collection, labeling, and training models from scratch. Transfer learning with pre-trained models can be used for AI applications in smart cities, retail, healthcare, industrial inspection and more. TAO supports training for CV, 3D Point cloud, ASR, NLP and TTS modalities.
TAO packages a collection of containers, python wheels, models and helm chart. AI training tasks run either on TensorFlow or PyTorch depending upon the entrypoint for the model.
For deployment, TAO models can be deployed to DeepStream for video analytics applications, Riva for Conversational AI applications or Triton for inference serving use cases.
TAO Containers
All containers needed to run TAO can be pulled from this location. See the list below for all available containers in this registry.
| TAO Container Type | container_name:tag | What's it used for? |
|---|---|---|
| TAO TensorFlow v1 container | nvcr.io/nvidia/tao/tao-toolkit:5.0.0-tf1.15.5 | Older CV networks like YOLOs, FasterRCNN, DetectNet_v2, MaskRCNN, UNET and more |
| TAO TensorFlow v2 container | nvcr.io/nvidia/tao/tao-toolkit:5.5.0-tf2 | CV networks like EfficientDet, EfficientNet and more |
| TAO PyTorch container | nvcr.io/nvidia/tao/tao-toolkit:5.5.0-pyt | Newer CV networks like Deformable-DETR, SegFormer and more as well as all ConvAI networks |
| TAO Deploy container | nvcr.io/nvidia/tao/tao-toolkit:5.5.0-deploy | Container used to TensorRT engine, INT8 calibration from a trained TAO model and evaluation on said TensorRT engine |
| TAO Data Service | nvcr.io/nvidia/tao/tao-toolkit:5.5.0-dataservice | Container for AI-assisted annotation and few other data services |
| TAO API container | nvcr.io/nvidia/tao/tao-toolkit:5.5.0-api | Front-end services container that can be used to host a TAO REST API server for remote execution of model training tasks. Useful for building higher level services |
How to run TAO?
To get started with TAO, download the sample tutorials from here and follow the quick start instructions.
License
TAO Toolkit getting Started License for TAO containers is included in the banner of the container. License for the pre-trained models are available with the model cards on NGC. By pulling and using the Train Adapt Optimize (TAO) Toolkit container to download models, you accept the terms and conditions of these licenses.
Technical blogs
- Access the latest in Vision AI development workflows with NVIDIA TAO Toolkit 5.0
- Improve accuracy and robustness of vision ai models with vision transformers and NVIDIA TAO
- Train like a ‘pro’ without being an AI expert using TAO AutoML
- Create Custom AI models using NVIDIA TAO Toolkit with Azure Machine Learning
- Developing and Deploying AI-powered Robots with NVIDIA Isaac Sim and NVIDIA TAO
- Learn endless ways to adapt and supercharge your AI workflows with TAO - Whitepaper
- Customize Action Recognition with TAO and deploy with DeepStream
- Read the 2 part blog on training and optimizing 2D body pose estimation model with TAO - Part 1 | Part 2
- Learn how to train real-time License plate detection and recognition app with TAO and DeepStream.
- Model accuracy is extremely important, learn how you can achieve state of the art accuracy for classification and object detection models using TAO
Suggested reading
- More information on about TAO Toolkit and pre-trained models can be found at the NVIDIA Developer Zone
- TAO documentation
- Read the TAO getting Started guide and release notes.
- If you have any questions or feedback, please refer to the discussions on TAO Toolkit Developer Forums
- Deploy your models for video analytics application using DeepStream. Learn more about DeepStream SDK
- Deploy your models in Riva for ConvAI use case.
Ethical AI
NVIDIA’s platforms and application frameworks enable developers to build a wide array of AI applications. Consider potential algorithmic bias when choosing or creating the models being deployed. Work with the model’s developer to ensure that it meets the requirements for the relevant industry and use case; that the necessary instruction and documentation are provided to understand error rates, confidence intervals, and results; and that the model is being used under the conditions and in the manner intended.