TrafficCamNet Transformer - Lite

Model Overview

Description

TrafficCamNet Transformer - Lite is an object-detection model that detects one or more objects from the following four categories in an image and returns a bounding box and category label for each detected object.

• car
• road sign
• person
• bicycle

This model is ready for commercial use.

GOVERNING TERMS:

Use of this model is governed by the NVIDIA Open Model License Agreement.

Deployment Geography:

Global

Use Case

This model can be used in computer-vision use cases to detect cars, road signs, people, and two-wheelers in video.

Release Date:

NGC [11/14/2025] via URL

Reference

• Y. Zhao, W. Lv, S. Xu, J. Wei, G. Wang, Q. Dang, Y. Liu, J. Chen: DETRs Beat YOLOs on Real-time Object Detection

Model Architecture

Architecture Type: Convolution Neural Network + Transformer Encoder Decoder. Network Architecture:

This model was developed based on the RT-DETR object detection model with a CNN backbone. The CNN in the backbone is a ResNet50 model, but the detection head is a Transformer encoder decoder.

** Number of model parameters: 4.5 * 10^7

Computational Load (Internal Only: For NVIDIA Models Only)

Cumulative Compute: 6.15 * 10^13
Estimated Energy and Emissions for Model Training: 2.5 * 10^3 KWH

Input

Input Type: Image
Input Formats: Red, Green, Blue (RGB)
Input Parameters: Two-Dimensional (2D)
Other Properties Related to Input: Minimum 32 x 32 Resolution required; no alpha channel or bits

Note: All model variants were fine-tuned with 3x544x960 (CxHxW) image input.

Output

Output Type(s): Bounding boxes and Class labels
Output Parameters: One Dimensional (1D), Two Dimensional (2D) vectors
Other Properties Related to Output:

• pred_logits: B x 300 (Batch Size x Number of Queries)
• pred_boxes: B x 300 x 4 (Batch Size x Number of Queries x Coordinates in cxcywh format)

Our AI models are designed and/or optimized to run on NVIDIA GPU-accelerated systems. By leveraging NVIDIA's hardware (e.g. GPU cores) and software frameworks (e.g., CUDA libraries), the model achieves faster training and inference times compared to CPU-only solutions.

How to Use This Model

This model requires NVIDIA hardware and software. Hardware requirements: any NVIDIA GPU with at least 12 GB of memory. Training and fine-tuning are supported with the TAO Toolkit.

The primary use case for this models is object detection.

The model is intended to be fine-tuned using the Train, Adapt, Optimize (TAO) Toolkit or used directly with deployment SDKs such as DeepStream, Triton, or TensorRT for object detection in traffic systems. High-fidelity models can be trained to detect classes not originally included in the model. A Jupyter notebook is available as part of the TAO container and can be used for retraining.

Instructions to Use Pretrained Models with TAO

To use these models as pretrained weights for transfer learning, use the following snippet as a template for the model, dataset and train components of the experiment spec file to train a RT-DETR model. For more information on the experiment spec file, see the TAO Toolkit User Guide.

train:
  pretrained_model_path: /path/to/trafficcamnet/resnet50_its.pth
  precision: 'bf16'
  checkpoint_interval: 10
  validation_interval: 10
  num_epochs: 100
model:
  backbone: resnet_50
  train_backbone: true
dataset:
  train_data_sources:
  - image_dir: /path/to/train/images
    json_file: /path/to/coco/format/train/annotations.json
  val_data_sources:
    image_dir: /path/to/validation/images
    json_file: /path/to/coco/format/val/annotations.json
  test_data_sources:
    image_dir: /path/to/test/images
    json_file: /path/to/coco/format/test/annotations.json
  infer_data_sources:
    image_dir:
    - /media/scratch.metropolis3/vpraveen/datasets/its_datasets/legacy_datasets_val/images
    classmap: /path/to/labels.txt
  batch_size: 4
  workers: 8
  remap_mscoco_category: false
  pin_memory: true
  dataset_type: serialized
  num_classes: 5
  eval_class_ids: null
  augmentation:
    multi_scales:
    - - 480
      - 832
    - - 512
      - 896
    - - 544
      - 960
    - - 544
      - 960
    - - 544
      - 960
    - - 576
      - 992
    - - 608
      - 1056
    - - 672
      - 1184
    - - 704
      - 1216
    - - 736
      - 1280
    - - 768
      - 1344
    - - 800
      - 1408
    train_spatial_size:
    - 544
    - 960
    eval_spatial_size:
    - 544
    - 960

Software Integration

Runtime Engine:

• TAO 6.0.0

Supported Hardware Microarchitecture Compatibility:

• NVIDIA Ampere
• NVIDIA Blackwell
• NVIDIA Hopper
• NVIDIA Lovelace
• NVIDIA Pascal
• NVIDIA Turing

[Preferred/Supported] Operating System(s):

• Linux

Model Versions

• trainable_resnet50_v2.0 - Pre-trained Resnet50 variant of the model to facilitate transfer learning via TAO Toolkit.
• deployable_resnet50_v2.0 - Pre-trained Resnet50 variant of the model that's optimized for deployment via TensorRT.
• trainable_resnet18_v2.0 - Pre-trained Resnet18 variant of the model to facilitate transfer learning via TAO Toolkit.
• deployable_resnet18_v2.0 - Pre-trained ResNet18 variant of the model that's optimized for deployment via TensorRT.

Training, Testing, and Evaluation Datasets:

Training Datasets

Link:

• ITS Dataset (NVIDIA Internal)

Data Modality

• Image

Image Training Data Size

• Less than a Million Images

Data Collection Method by dataset:

• Hybrid: Automated, Human (custom collected and curated)
  

Labeling Method by dataset:

• Hybrid: Automated, Human (custom collected and curated)
  

Properties:

Testing Datasets

Link:

• ITS Dataset (NVIDIA Internal)

Data Modality

• Image Data Collection Method by dataset:
  
• Hybrid: Automated, Human (custom collected and curated)
  

Labeling Method by dataset:

• Hybrid: Automated, Human
  

Properties:

Evaluation Datasets

Link:

• ITS Dataset (NVIDIA Internal)

Data Collection Method by dataset:

• Hybrid: Automated, Human (custom collected and curated)
  

Labeling Method by dataset:

• Hybrid: Automated, Human
  

Properties:

Performance

Evaluation Data

We evaluated the TrafficCamNet Transformer - Lite models on a curated dataset of 18,500 proprietary images across a variety of environments and traffic intersections, with corresponding bounding-box annotations for cars, road signs, people, and bicycles. These frames are high-resolution 1920x1080 images, resized to 960x544 before running inference with the model.

Methodology and KPI

We compute mean Average Precision at IoU = 0.50 (mAP50). The table below reports per-class mAP50 across multiple datasets/conditions. Values are in [0, 1].

Inference

Acceleration Engine: Tensor(RT), DeepStream

The TrafficCamNet Transformer - Lite model inference can be run via DeepStream using the same tao_detection apps.

Test Hardware:

• A2
• A30
• DGX H100
• DGX A100
• L4
• L40
• Orin
• Orin Nano 8GB
• Orin NX
• Orin NX16GB

Inference is performed on the provided unpruned model at FP16 precision. Inference performance is measured using trtexec on Jetson AGX Xavier, Xavier NX, Orin, Orin NX, NVIDIA T4, and Ampere GPUs. Jetson devices run in the Max-N configuration to achieve maximum GPU frequency. The performance shown here reflects inference-only results. End-to-end performance with streaming video may vary depending on hardware and software bottlenecks.

Ethical Considerations

NVIDIA believes Trustworthy AI is a shared responsibility and we have established policies and practices to enable development for a wide array of AI applications. When downloaded or used in accordance with our terms of service, developers should work with their internal model team to ensure this model meets requirements for the relevant industry and use case and addresses unforeseen product misuse. For more detailed information on ethical considerations for this model, please see the Model Card++ Bias, Explainability, Safety & Security, and Privacy Subcards. Please report model quality, risk, security vulnerabilities or NVIDIA AI Concerns here.