Jetson Platform Services Reference Workflow & Resources
Jetson Platform Services
What is Jetson Platform Services?
NVIDIA Jetson Platform Services, an integral component of NVIDIA JetPack™ SDK, gives you a collection of powerful cloud-native microservices and building blocks to build powerful vision AI applications for the edge powered by NVIDIA Jetson™. Jetson Platform Services simplifies development, deployment and management of Edge AI applications for NVIDIA Jetson by providing a modular and extensible architecture for developers to distill large complex applications into smaller modular microservice with APIs to integrate into other apps and services.
Elevate your application by seamlessly integrating generative AI capabilities and future-proof your solutions through a sophisticated blend of microservices, providing an API-driven, modular, and highly extensible framework.
This latest release of NVIDIA Jetson Platform Services includes:
- Reference workflows for: AI-NVR, Zero Shot Detection and Vision Language Models (VLM)
- 15+ services, including generative AI services, foundation services, and more
- Streaming generative AI services for vision language model and zero-shot detection with APIs for dynamic prompting and stream update
- DeepStream-based AI perception service for object detection and tracking with APIs for stream update
- Media management and storage service with support for streaming protocol such as RTSP and WebRTC
- Core foundation services like IoT Gateway, API Gateway, monitoring, and system bus
- IoT services for secure edge-to-cloud connectivity and remote API invocation
How to Get Started?
Hardware Requirements:
- Jetson Orin AGX devkit or
- Jetson Orin NX 16GB or 8GB devkit (self built) with 128GB(min) NVMe drive or
- Jetson Orin Nano 8GB devkit with 128GB(min) NVMe drive or
- Partner (ODM) device with one of above
- Ubuntu 20.04 or 22.04 Desktop/Laptop
- USB-C flashing cable
- Monitor, Keyboard, Mouse
For full requirement and hardware setup, refer to the Preparation and Hardware Setup in the Quick Start Guide.
Software Requirements:
- JetPack 6.0/Jetson Linux 36.3
- Jetson Platform services
The only requirement to use Jetson Platform Services is JetPack 6.0/Jetson Linux 36.3. For full instructions on flashing the device with JetPack 6.0, please refer to Software Setup in Quick Start Guide. Follow instructions all the way down to Installing Jetson Platform Services.
Download and Extract Bundle:
Download the reference workflow and resource bundle from this page as a starting point. This bundle will include the resources required to run the reference workflows for the Hello World example, AI-NVR, Zero Shot Detection and VLM.
Run Hello World Example:
Continue following the Quick Start Guide through the Setup steps and Hello World example to get familiar with Jetson Platform Services.
This Hello World example will show to run an Intelligent Video Analytics application that will uses RTSP streams generated by NVStreamer, draw trip wires and regions of interest with the Video Storage Toolkit and automatically inference the streams with DeepStream and generate analytic data.
Advanced Workflows:
To continue with the more advanced workflows for AI-NVR and generative AI also found in this resource bundle, view the pages below:
Package Content: This NGC resource contains several tar files.
The following packages contain docker compose and configuration files needed to run each workflow or microservice.
- ai_nvr-1.1.0.tar.gz
- zero_shot_detection-1.1.0.tar.gz
- vlm-1.1.0.tar.gz
- nvstreamer-1.1.0.tar.gz
Additionally some starter resources are also includes such as sample videos and reference scripts for cloud deployment.
- ai-nvr-app-android-1.1.0.tar.gz
- jps-cloud-scripts-1.1.0.tar.gz
- jps-cloud-terraform-1.1.0.tar.gz
- sample_1080p_h264.mp4
Navigating all the resources on NGC
All the resources are provided on NGC.
| Category | Name | Description |
|---|---|---|
| Resources | Jetson Platform Services Reference Workflow & Resources | Starting resources for Jetson Platform Services. Includes sample worfklows, videos and scripts. |
| Resources | Jetson Platform Services OSS Sources | Package that includes all third part OSS source code and licenses used across all Jetson Platform Services. |
| Container | ialpha-ingress-arm64v8 | A reverse proxy that forwards incoming API requests to the appropriate microservice |
| Container | its-monitoring | Used to monitor, persist, and visualize edge device status |
| Container | jps-metric-exporter | Used to monitor system metrics and export to Promtheus for tracking |
| Container | tcpmux-client | Enables cloud to connect to the device, thereby allowing clients to invoke device APIs remotely |
| Container | prov-agent | Procures PKI assets needed by the device to connect securely to the cloud. |
| Container | notify-hub | A notification service used to forward analytic notifications to external devices |
| Container | vst:x_aarch64 | Video Storage Toolkit microservice that enables discovery, storage and streaming of video sources |
| Container | vst:nvstreamer_x_aarch64 | NVStreamer container built for Jetson. Used to serve video files as RTSP streams. |
| Container | vst:nvstreamer_x_x86_64 | NVStreamer container built for x86. Can be used on x86 device to serve video files as RTSP streams. |
| Container | emdx-analytics | Analytics service to create count analytics like line-crossing, Region of Interest (ROIs) and Field of View (FOV) |
| Container | emdx-analytics-web-api | Provides APIs to interact with the Analytics service |
| Container | sdr | Microservice to dynamically add & register streams to Deepstream, Analytic and Generative AI conatiners |
| Container | sdr-reprovision-controller | Microservice to ensure SDR and VST stream lists are in sync |
| Container | deepstream | AI perception service with pre-built DeepStream pipeline and PeopleNet model for People Detection |
| Container | zero_shot_detection_infer | Zero Shot Detection AI service that performs open vocabulary detection on live streams and is controllable through REST APIs. |
| Container | vlm_infer | Vision Language Model (VLM) AI service that allows VLM inference on live stream with custom user prompts and alerts controllable through REST APIs. |
| Container | device-proxy | Provides APIs to proxy requests to device via tcpmux |
| Container | tcpmux-server | Service for IoT Gateway - implements always-on device to cloud connection, thereby allowing clients to invoke device APIs remotely |
| Container | db-migration | Configure the database for reference cloud |
| Container | authorizer | Validates users by working with the Identity Provider (IDP) service |
| Container | client-rest-api | Provides APIs for User profile management, device profile management, User-device mapping, User-device claim |
| Container | cloud-proxy | Entry point for cloud uses to access device bound APIs |
| Container | prov-server-agent-api | Cloud microservice to issue unique certificates to devices |
| Container | prov-server-admin-api | Cloud microservice to provide OTP (one time passcode) used by devices to request certificates |
| Container | cloud-gateway | Entry point for cloud users to access cloud bound APIs |
| Container | notification-handler | Service that ingests emdx alerts and pushes them to AWS SNS for cloud integration. |
| Container | device-gateway | Web proxy service that receives emdx alerts and forwards them to the notification-handler service. |
Developer Resources
Documentation - Jetson Platform Services
Documentation - Quick Start Guide
Blog - Bringing Generative AI to the Edge
Blog - Build Vision AI Applications with APIs
Technical Whitepaper on Microservices for Jetson
Webinar Part 1 - Accelerate Edge AI Development With Metropolis APIs and Microservices on Jetson
Webinar Part 2 - How to Build With Metropolis Microservices for Jetson
Learn more about Metropolis Microservices
License
By downloading or using the software and materials, you agree to the License Agreement for JetPack.