TAO Non-Commercial Pretrained FasterViT Classification Model

What is Train Adapt Optimize (TAO) Toolkit?

Train Adapt Optimize (TAO) Toolkit is a Python-based AI toolkit for customizing purpose-built pre-trained AI models with your own data. TAO Toolkit adapts popular network architectures and backbones for your data, allowing you to train, fine-tune, prune, and export highly optimized and accurate AI models for edge deployment.

The pre-trained models accelerate the AI training process and reduce costs associated with large-scale data collection, labeling, and training from scratch. Transfer learning with pre-trained models can be used for AI applications in smart cities, retail, healthcare, industrial inspection, and more.

Build end-to-end services and solutions for transforming pixels and sensor data into actionable insights using the TAO DeepStream SDK and TensorRT. These models are suitable for object detection, classification, and segmentation.

Model Overview

Image Classification is a popular computer vision technique in which an image is classified into one of the designated classes based on the image features. This model card contains pretrained weights for most of the popular classification models. These weights can be used as a starting point with the classification app in TAO Toolkit to facilitate transfer learning.

Model Architecture

FasterViT is a hybrid CNN-ViT-based family of backbone from NVIDIA research that achieves SOTA in ImageNet-1K classification with focus on throughput. This family of backbone leverages Hierarchical Attention (HAT), which decomposes global self-attention with quadratic complexity into a multi-level attention with reduced computational cost. FasterViT architecture was designed with TensorRT in mind, so the model throughput is highly optimized in TensorRT SDK. Use FasterViT when you want to achieve SOTA accuracy on your target dataset while achieving higher throughput compared to other Vision Transformers such as Swin and ConvNext.

Training

This model was trained using the classification_pyt entrypoint in TAO. The training algorithm optimizes the network to minimize cross-entropy loss.

Training Data

Most of the FasterViT models were trained on ImageNet1K dataset. FasterViT-Large was trained on ImageNet22k dataset

Performance

Evaluation Data

The FasterViT models have been evaluated on the ImageNet1K validation dataset.

Methodology and KPI

The key performance indicator is accuracy, following the standard evaluation protocol for image classification. The KPI for the evaluation data are:

\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n