NGC | Catalog Replicator TAO Replicator TAO

Logo for Replicator TAO
Jupyter Notebook for training with TAO using Omniverse Replicator sythetic data generated on
Latest Version
March 4, 2024
Compressed Size
12.48 KB

NVIDIA TAO with Omniverse Replicator Synthetic Data via's anatools SDK

The ana_replicator_tao resource contains a Jupyter Notebook and supporting configuration files to train the TAO DetectNet_v2 Object Detection model using Omniverse Replicator Synthetic Data generated on the Platfrom. A user applies transfer learning to a Resnet-18 backbone with a targeted dataset to achieve an object detection model that is evaluated on Replicator based datasets for various occlusion levels. The notebook demonstrates the effectiveness of the TAO Toolkit,'s anatools, and Replicator to train CV models.

Model Training and Evaluation with NVIDIA TAO and

Using This Resource

The notebook is best run in a Python environment for the TAO Toolkit Launcher. For the requirements, see the TAO Toolkit Quick Start Guide.

Traing and evaluation datasets are retrieved from the user's account. Register for a free trial with to get started. From the platform, create a new workspace with the content code REPLICATOR4TAO. This will load up a workspace for you that has the graphs used in the notebook.

Replicator Human Occlusion Workspace

Occlusion Experiment

Once the transfer learning is complete, the model is evaluated by retrieving datasets from your organization. You can adjust the graphs to generate datasets with various occlusion levels by adding more shrub props.

Medium Occlusion Graph

The notebook has a function to estimate the average occlusion for any dataset from the occlusion value in the KITTI label. See the NVIDA Docs for a description of the Occlusion KITTI Parameter.

Effect of Occlusion on the TAO DetectNet_v2 Model

Finally, you can inference specific images to visualize the model's performance. Spot checking a batch of inferences can reveal biases in the model. With closed loop ML, you can use to update the Replicator parameters, generate more training data and re-evaluate the performance.

TAO Inference on images generated with Omniverse Replicator

About is a platform-as-a-service for synthetic data that enables data scientists to overcome the costs and challenges of acquiring and using real data for training machine learning and artificial intelligence systems.

Check us out: