NVIDIA's Transfer Learning Toolkit Computer Vision Inference Pipeline allows developers to train and deploy purpose-built AI models and build AI applications using a C++ based API.
TLT Computer Vision Inference Pipeline for L4T
The Transfer Learning Toolkit (TLT) Computer Vision Inference Pipeline for L4T requires several containers:
- NVIDIA Triton Inference Server
- Server Utilities for hosting TLT models
- Client applications and samples using the TLT CV API
These containers are specifically built for NVIDIA Jetson devices running Jetpack with Linux for Tegra (L4T). Please check the Requirements and Installation before usage.
The NVIDIA Triton Inference Server built for L4T is provided through GitHub. This exists as another process and serves inferences to the Client container, which houses applications and sample usage for the TLT Computer Vision API.
The Server Utilities container contains the folder structure and libraries necessary for the NVIDIA Triton Inference Server to serve inferences. This container also allows for simple TLT model conversion from the TLT Computer Vision Quick Start.
These containers and tags will automatically be pulled by the Quick Start Scripts without intervention from the user. Please check the guidelines provided by the Quick Start Scripts for more details.
Client Container
The Client Container provides an environment with the TLT CV Inference Pipeline libraries and open-source demos that enable developers to build and deploy custom applications.
The included demos are as follows:
- Body Pose Estimation
- Emotion Classification
- Face Detection
- Facial Landmark Estimation
- Gaze Estimation
- Gesture Recognition
Each of these demos leverages appicable Transfer Learning Toolkit models, which can be retrained.
These open source demos highlight the C++ API that allows for inference requests
(eg. getBodyPose(), getGaze()).
These can run using a webcam device or a video file/stream. There is also an API that allows for custom image decode. More information is provided in the demo source and the API Documentation in the Quick Start. Further configuration documentation is also provided through the Transfer Learning Toolkit Documentation.
Some examples of applications that one can build are event-based applications. If a person is providing a thumbs-up gesture or is looking directly at the camera, the gesture recognition and gaze estimation APIs can allow for the detection of these events.
License
License for TLT Computer Vision Inference Pipeline containers is included within the containers at workspace/TLT-CV-Inference-Pipeline-EULA.pdf. License for the pre-trained models are available with the model files. By pulling and using the Transfer Learning Toolkit SDK (TLT) Computer Vision Inference Pipeline and downloading models, you accept the terms and conditions of these licenses.
Ethical AI
NVIDIA's platforms and application frameworks enable developers to build a wide array of AI applications. Consider potential algorithmic bias when choosing or creating the models being deployed. Work with the model's developer to ensure that it meets the requirements for the relevant industry and use case; that the necessary instruction and documentation are provided to understand error rates, confidence intervals, and results; and that the model is being used under the conditions and in the manner intended.