USD Code API

USD Code API is a Mixture of Agent API that leverages multiple agents built on top of Llama 3.1 LLM foundation model. It includes three experts:

1. Python-USD knowledge expert: Capable of answering OpenUSD knowledge questions
2. Core Python-USD code generation expert: Capable of generating Python-USD code in response to text prompts
3. High-level code generation expert: Capable of performing complex Python-USD stage modification leveraging Helper Functions.

USD Code API boosts your productivity when working with OpenUSD. Whether you are a highly skilled workflow developer or domain specific developer, USD Code API will enable you to learn and develop with OpenUSD more efficiently. With USD Code API you can accelerate common 3D development needs, such as synthetic data generation for robotics and computer vision applications. Learn more about OpenUSD here.

For more information please refer to USD Code API documentation.

1.0.0 - Feature Branch (GA) Release

Release Highlights

🔥 Feature Branch (GA) release - USD Code API

USD Code API 1.0.0 is a general availability release of USD Code API featuring various updates and improvements across all components. With this release, USD Code API added a new capability - ability to generate USD Code Helper Functions (High-level Python USD Code) to perform stage modification tasks.

Getting Started

• Helm chart installation documentation
• General information

The full list of new features and capabilities is outlined below.

Release Notes

1.0.0 GA

Release Date: December 2024

• Feature Branch (GA) release of USD Code API, including a downloadable Helm chart for self-deployment.

Added

• Added self-deployment option to integrate API into IDE and Omniverse Kit.
• Added ability to generate USD Code Helper Function (High-level Python USD Code) to perform stage modification tasks.

Improved

• Improved response quality:
  • Better contextual awareness and more relevant follow-up responses due to increased context window size.
• Improved developer experience in Kit 106.5 by providing clearer error messaging when API errors occur
• Improved expert routing robustness to ensure the right expert being routed to address user prompt tasks.

Known Limitations

• IDE Integration: Copilot capabilities in VS Code are limited by the Continue.dev extension's capabilities. Autocompletion is currently not supported by USD Code.
• Context window length: USD Code is based on a large language model that has a finite context window, so time-outs can occur if chats grow too long over the context window. As soon as previous chat history is no longer relevant to provide context, starting a new chat is recommended.

System Requirements

Deployment

A suitable setup example for running USDCode models efficiently is a Microk8s cluster configured with 5 (4+1) H100 GPUs enabled. By meeting these system requirements, you'll be well-equipped to deploy and utilize your USDCode models successfully.

• Hardware
  • 4 H100 / A100 GPUs or 8 H100 / A100 / L40S GPUs with 158GB of disk space (please refer to Support Matrix for llama 3.1-70b)
  • Any NVIDIA GPU with at least 2GB and 17GB of disk space for the embedding model (H100/A100/L40S/A10G/L4 for optimized configuration. Please refer to Support Matrix for nv-embed-e5-v5)
• Software
  • Linux operating systems (Ubuntu 20.04 or later recommended)
  • NVIDIA Driver >= 560
  • NVIDIA Docker >= 23.0.1
  • CUDA >= 12.6.1

Llama 3.1 -70b:1.3.0 is based on CUDA 12.6.1 which requires NVIDIA Driver release 560 or later. However, if you are running on a data center GPU (for example, A100 or any other data center GPU), you can use NVIDIA driver release 470.57 (or later R470), 535.86 (or later R535), or 550.54 (or later R550). Nv-embed-e5-v5 from 1.0.0 uses Triton Inference Server 24.05.

VS Code / Other IDE Integration

• Software
  • Required: Visual Studio Code
  • Required: Continue.dev (or your preferred copilot extension)
  • Required: Python 3.10

Sample Kit Extension

• Hardware
  • GPU: NVIDIA RTX capable GPU (Turing or newer recommended)
• Software
  • Operating System: Windows 10/11 or Linux (Ubuntu 20.04/22.04 recommended)
  • Driver: Please follow Omniverse developer guide - Technical Requirements section to ensure the correct versions are used.
  • Internet Access: Required for downloading the Omniverse Kit SDK, extensions, and tools.
  • Software Dependencies:
    • Required: Git (with LFS enabled)
    • Recommended: VS Code (or your preferred IDE)

LICENSE

If you download the software and materials as available from the NVIDIA AI product portfolio, use is governed by the NVIDIA Software License Agreement and the Product-Specific Terms for NVIDIA AI Products; except for the model which is governed by the NVIDIA AI Foundation Models Community License Agreement, and the RAG dataset which is governed by the terms of the NVIDIA Asset License.

ADDITIONAL INFORMATION: For Llama model, Llama 3.1 Community License Agreement, Built with Llama; for NV-EmbedQA-E5-v5: MIT license; for NV-EmbedQA-Mistral7B-v2: Apache 2.0 license, and Snowflake arctic-embed-l: Apache 2.0 license.

If you download the software and materials as available from the NVIDIA Omniverse portfolio, use is governed by the NVIDIA Software License Agreement and the Product-Specific Terms for NVIDIA Omniverse; except for the model which is governed by the NVIDIA AI Foundation Models Community License Agreement, and the RAG dataset which is governed by the terms of the NVIDIA Asset License.

ADDITIONAL INFORMATION: For Llama model, Llama 3.1 Community License Agreement, Built with Llama; for NV-EmbedQA-E5-v5: MIT license; for NV-EmbedQA-Mistral7B-v2: Apache 2.0 license, and Snowflake arctic-embed-l: Apache 2.0 license.