Linux / amd64
TAO (Train Adapt Optimize) Toolkit is a python based AI toolkit that's built on TensorFlow 2.0 and PyTorch. It provides transfer learning capability to adapt popular neural network architectures and backbones to your data, allowing you to train, fine-tune, prune, quantize and export highly optimized and accurate AI models for edge deployment. You can use TAO Toolkit to distill knowledge from large foundation models like C-RADIOv3 and ConvNext-L's, to smaller compute friendly models for the edge.
The purpose built pre-trained models accelerate the AI training process and reduce costs associated with large scale data collection, labeling, and training models from scratch. Transfer learning with pre-trained models can be used for AI applications in smart cities, retail, healthcare, industrial inspection and more. TAO supports training for CV and 3D Point cloud modalities.
TAO packages a collection of containers, python wheels, models and helm chart. AI training tasks run either on TensorFlow or PyTorch depending upon the entrypoint for the model.
For deployment, TAO models can be deployed to DeepStream for video analytics applications, or Triton for inference serving use cases.
Use of this software is governed by the NVIDIA Software License Agreement and Product-Specific Terms for NVIDIA AI Products. Your use of models with this software is subject to the terms accompanying the models.
TAO 6.0.0 FTMS includes the following assets:
All containers needed to run TAO can be pulled from this location. See the list below for all available containers in this registry.
| TAO Container Type | container_name:tag | What's it used for? |
|---|---|---|
| TAO PyTorch container | nvcr.io/nvidia/tao/tao-toolkit:6.0.0-pyt | Finetuning workflows in PyTorch |
| TAO TensorFlow2 container | nvcr.io/nvidia/tao/tao-toolkit:6.0.0-tf2 | Finetuning workflows in TensorFlow 2.0 |
| TAO Deploy container | nvcr.io/nvidia/tao/tao-toolkit:6.0.0-deploy | TensorRT inference workflows in Deploy |
| TAO Data Services container | nvcr.io/nvidia/tao/tao-toolkit:6.0.0-pyt | Dataset augmentation, autolabelling and analysis workflows |
To get started with TAO, download the tutorial notebook using the NGC CLI and run the following commands.
Note: The instructions to install the NGC CLI are detailed here.
git clone https://github.com/NVIDIA/tao_tutorials.git
git checkout release/6.0.0
You can run the kick-start the notebook by running
python3 -m pip install nvidia-tao==6.0.0
python3 -m pip install jupyter
jupyter notebook --ip 0.0.0.0 --port <port_number> --allow-root
The TAO 6.0.0 FTMS package refers several pre-trained models released as part of NGC.
| Model Name | Pulled | Use Case |
|---|---|---|
| CRADIOv2 | Manual | Multi-teacher distilled foundation model generating rich visual embeddings |
| ConvNextv2 | Manual | FC-MAE trained foundation model generating rich visual embeddings for CNNs |
| TrafficCamNet Transformer Lite | Manual | Object detection network for detecting 4 class objects in traffic scenes |
| DeiT-base | Automatic | Pretrained feature network for Projected GAN |
| EfficientNet | Automatic | Pretrained feature network for Projected GAN |
| DeiT-small | Automatic | Classifier guidance to inject class information |
| InceptionNet | Manual | Calculating FID scores/metrics |
| EfficientNet-embeddings | Manual | Prevent class embeddings collapse for optimization of the embeddings |
DeiT-base, EfficientNet, and DeiT-small are pulled automatically through the timm API in the training code. C-RADIOv2, InceptionNet and EfficientNet-embeddings need to be pulled manually. The tutorial notebooks outline the instructions and steps to pull these models.
NVIDIA’s platforms and application frameworks enable developers to build a wide array of AI applications. Consider potential algorithmic bias when choosing or creating the models being deployed. Work with the model’s developer to ensure that it meets the requirements for the relevant industry and use case; that the necessary instruction and documentation are provided to understand error rates, confidence intervals, and results; and that the model is being used under the conditions and in the manner intended.
Please review the Security Scanning (LINK) tab to view the latest security scan results. For certain open-source vulnerabilities listed in the scan results, NVIDIA provides a response in the form of a Vulnerability Exploitability eXchange (VEX) document. The VEX information can be reviewed and downloaded from the Security Scanning (LINK) tab.