USD Search API

USD Search API is a collection of cloud-native microservices that enable developers, creators, and workflow specialists to efficiently search through vast collections of OpenUSD data, images, and other assets using natural language or image-based inputs.

With these production-ready microservices, developers can deploy USD Search API onto their own infrastructure. With USD Search API’s artificial intelligence (AI) features, you can quickly locate untagged and unstructured 3D data and digital assets, saving time navigating unstructured, untagged 3D data. USD Search API is capable of searching and indexing 3D asset databases, as well as navigating complex 3D scenes to perform spatial searches, without requiring manual tagging of assets.

For more information please refer to USD Search API documentation.

1.4 - GA Release

Release Highlights

🔥 Explorer UI Refactor brings a redesigned user experience, advanced filter rail, and asset download manager to the sample Explorer application

🔥 Search and Deployment Skill Improvements — improved Claude Code and Codex skills for search and deployment workflows

🔥 NEW LLM-based query understanding (beta) parses free-text queries into structured search filters for more precise results

🔥 NEW Asset loading functionality (beta) bundles an indexed asset and its transitive dependencies into a self-contained ZIP archive, with a pre-flight size preview

Getting Started

• Collection of scripts that simplify installation of USD Search on a kubernetes cluster.
• Helm chart installation documentation
• GitHub

The full list of new features and capabilities is outlined below.

Release Notes

1.4.0 GA

Release Date: July 2026

• Asset download, Explorer web UI refactor, storage API backend support, and consolidated LLM access

Added

• Asset Download API — GET /download/asset bundles an indexed asset and its transitive dependencies into a self-contained ZIP archive, with a pre-flight size preview and a manifest listing included and skipped files
• Explorer Web UI Refactor — redesigned user experience with an advanced filter rail, asset download manager, zoomable asset preview and dependency graph, a discoverable filter dropdown listing the deployment's supported filters, and optional LLM-based query parsing that turns free text into removable filter chips (falling back to plain hybrid search when disabled or unavailable)
• Shared LLM Client Package — one OpenAI-compatible client for every AI role, configurable via USDSEARCH_LLM_API_KEY and USDSEARCH_LLM_BASE_URL

Updated

• Search and Deployment Skills — improved Claude Code and Codex skills for search and deployment workflows
• Skill and quickstart scripts update to support Storage API Backend
• LLM and VLM Access — consolidated behind the shared LLM client, replacing multiple provider specific integrations
• Storage Client Performance — parallelized thumbnail retrieval and reduced redundant storage listing calls
• Search Response Payload — reduced response size by returning only required embedding fields
• GIF Thumbnail processing - added configurable gif_sampling_mode option that supports fixed (every Nth frame) or uniform (frames spread evenly) sampling

Fixed

• Rendering Service with Storage API Backends — the rendering service now works with Storage API backends, including bearer token authentication, virtual hosted style HTTPS URL parsing for access control checks and downloads, and a working storage info endpoint
• Search Query Filtering — malformed property filters degrade to semantic search instead of filtering out all results
• Spatial Search Validation — invalid radius values now return a proper error response
• API Route Handling — trailing slash paths return not found instead of redirecting

Security

• Cache Deserialization — Redis and file cache payloads now use an allow list based deserializer, closing a remote code execution vector
• Download Logging — request derived values are sanitized before logging, preventing log injection
• API Gateway CORS — configured allowed origins now enforced across all endpoints

Known Limitations

• USD Search API relies on Omniverse Kit for rendering and extracting scene prim hierarchy. Therefore the service should work with any scene that Omniverse Kit can work with.
• When using the filter_by_properties query parameter in USD Search API, different filters must be separated by a single comma (having a comma and a space is not supported).
• Pagination of search results is currently not supported.

1.3.2 GA

Release Date: May 2026

• Initial open source release of USD Search API

Added

• Open Source Release — USD Search API source code, including Claude Code and Codex skills for search, scene inspection, asset inspection, deployment, and quickstart

1.3.1 GA

Release Date: May 2026

• Helm chart documentation improvements, rendering service improvements, and bug fixes

Added

• Configurable OpenSearch serializer via global.search_backend_config.serializer setting (default: orjson)
• Search REST API HPA autoscaler
• Documentation on using rendering service as an alternative to rendering jobs
• Thumbnail indexing settings configuration documentation
• Maintenance mode for deploying the helm chart without background indexing (for Redis queue cleanup and other database maintenance)
• rendering-service.settings.load_asset_as_reference — USD assets with a default prim are referenced into the scene rather than opened as a sublayer; falls back to sublayer when no default prim is defined. Available for both the rendering service and Kubernetes rendering job.
• Configurable rendering-service.settings.kit_extra_args and rendering-service.loggingConfig from the main chart values.yaml
• Search Backend Readiness Init Container — added to all services that depend on OpenSearch, ensuring the backend is ready before dependent containers start

Fixed

• Query Results Validation with Image Prompt — query results validation was not working properly with an image prompt
• Rendering Service Ephemeral Storage Exhaustion — rendering service was writing cache to an internal location, causing ephemeral storage to be overloaded and containers to be evicted

Updated

• Default settings for the rendering service temporary directory
• API gateway configuration to expose the search validation endpoint

Removed

• Memcached dependency — no caching performance improvement observed for rendering jobs and rendering service
• Deprecated storage service interface for OpenSearch backend

1.3.0 GA

Release Date: March 2026

• NEW SigLIP2 embedding model significantly improve search quality comparing to the previously used one
• NEW Query relevance verification functionality allows to do post-search evaluation of how relevant search results are with respect to input query using a VLM. This significantly improve search quality especially in cases of domain specific queries and when the object of interest is under-represented in the database. (NOTE: VLM access needs to be configured at deployment time)
• Expanded USD properties search functionality to allow filtering for numeric properties and added support for indexing polygon count for USD assets
• NEW Storage API backend support has been added to allow interacting with Omniverse Storage APIs

Added

• switch to using SigLIP2 as a based embedding model.
• added the possibility to enabled VLM-based results relevance check.
• added support for processing multiple thumbnails for the asset
• support for broader set of VLM models including (Google Gemini, Azure OpenAI, etc.)
• support for limiting chunks size of search backend bulk operations
• added support for filtering by polygon count
• Add support for >, >=, <, <= queries on numeric fields extracted from USD metadata
• added support for relying on rendering service instead of dynamic k8s rendering jobs (currently in beta)
  • added HSSC cache for rendering service
  • added the possibility to set the limit of how much RAM memory each Kit process may occupy in order to not crash the worker
  • added the possibility to reject further requests by the rendering service to avoid queue from filling up
• added support to enable CORS on the API gateway level
• added admin tools to allow cleaning up processing queues

Updated

• default OpenSearch and Redis settings
• removed some of the services that were used for metrics collection
  • moved some metric collection from service monitors to pod monitors
• OpenSearch version is updated to 3.3.2
• Helm chart dependency updates

Fixed

• support rendering assets that do not have default prim explicitly defined
• fixed the issue when passing invalid token still allowed searching
• removed mentions of deprecated azure storage backend type to avoid confusion, connecting to Azure blobstorage is fully supported through s3proxy functionality
• fix possible image pull secret duplication issue (that throws error with helm>4)
• removed kube-dns resolver from API gateway config
• s3 re-scan timeout is not properly applied
• tag based filtering
• updated neo4j helm chart and livenessprobe to check if the database is operational

NOTE: Since the base embedding model has been updated to SigLIP2, previously extracted embeddings are not compatible - the background re-indexing process will start automatically after the instance update.

1.2.0 GA

Release Date: September 2025

• Hybrid search functionality that allows to simultaneously search across vector and textual fields
• Images API that allows retrieving image for assets that have been rendered by the service
• Sample Web UI that allows to quickly experiment with USD Search APIs

Added

• Hybrid Search functionality
• Images API to retrieve rendered images for assets
• Possibility to skip Nucleus mounts
• Rendering job configuration settings in the helm chart configuration
• Authentication mechanism for embedding service allowing it to run on NVCF
• Embedding service API test in the helm chart
• Affinity support in rendering Job template
• Possibility to pull the explorer sample WebUI from git repository
• Admin tools (beta) that allow some basic operations (e.g. bulk re-indexing of asset).

Fixed

• Embedding liveness check does not work with NVCLIP NIM model
• Service names for plugin worker metrics are too long
• Added mapping verification to make sure usd_properties field exits in the index with the nested type
• Allow the possibility of passing numeric values as credentials
• S3proxy configuration bugfix
• VLM-based auto-captioning bugfix when default prompt and metadata fields could not be modified
• Security updates and vulnerability fixes

Migration from previous versions of USD Search

NOTE: Please ignore this section if you have not been using VLM-based auto-captioning functionality.

In 1.2.0 version of USD Search API, we have updated the way VLM-generated metadata is stored in the search index. If you have been using VLM-base auto-captioning functionality - you need to trigger re-indexing of VLM-based plugins for all assets on the storage backend. This can be done with the following command (after the helm chart is successfully deployed):

kubectl exec -it $(kubectl get pods -o name | grep deepsearch-info-endpoint | head -n 1) -- \
    python -m usdsearch.admin.tools reindex \
    --path="/" \
    --include_plugins="['thumbnail_to_vision_metadata', 'rendering_to_vision_metadata', 'image_to_vision_metadata']" \
    --ignore_existing_statuses

1.1.1 GA

Release Date: May 2025

• Helm chart fixes and documentation improvement

Added

• Indexing of custom USD properties documentation improvement
• Added a guide on how Nucleus access token can be used instead of the service name / password pair.
• Added a guide on how storage backend credential secrets could be manually created to avoid passing credential information through helm chart values.
• Added documentation section on how VLM prompt and metadata fields could be configured
• Added storage connection check before rendering job is created to avoid spinning up full Kit instance if storage backend becomes unaccessible for some reason
• Added FAQ section about Prometheus metrics missing in Grafana dashboard
• Added the possibility to limit the maximum number of parallel Kit instances created as part of rendering jobs. This is specifically useful for Nucleus storage backend, as Nucleus may not be able to handle arbitrary load.
• Described USD Search plugins in more details.
• Added a description on how to clean leftover rendering jobs after helm uninstall.
• Added best practices for organizing data on the storage backend.
• Added best practices for image-based search quality improvement

Fixed

• S3proxy configuration bugfix.

1.1.0 GA

Release Date: February 2025

• VLM-based auto-captioning and auto-tagging.
• Support for a broad range of storage backends through S3proxy functionality.
• Search by custom metadata fields extracted from USD assets.

Added

• VLM-based automatic tagging and captioning.
  • allows configuring USD Search to use a broad variety of VLM providers (e.g. NVIDIA AI Endpoints, OpenAI, Azure OpenAI, Mistralai, Anthropic)
• Support for a broad variety of backends through S3proxy.
  • for convenience the helm sub-chart for s3proxy configuration is provided.
• Filter search results by applying regex on Asset URLs.
• Search by custom metadata fields extracted from USD files.
  • Added support for searching by metadata stored in the customLayerData dict.
• On-demand indexing endpoint that allows to prioritize processing of selected assets without the need to wait for the background job to catch up.
• Make sure size search is happening over bounding box dimensions that are scaled by MPU.

1.0.0 GA

Release Date: November 2024

• GA release of USD Search API, including a downloadable Helm chart for self-deployment to interface with your own data.
• Beta feature added - Asset Graph Search to enable In-Scene Search, Spatial Search, USD Property Queries, Asset Dependency Tracking, Combined Query Capabilities .
• Added Python Client Library available in GitHub for ease of development.

Added

• Added Asset Graph Search [Beta] new feature
  • In-Scene Search
    • Locate specific assets within scenes using natural language or image similarity.
  • Spatial Search
    • Execute proximity queries using coordinates or relative to specific prims.
    • Find objects within a specified bounding box or radius.
    • Results can be sorted by distance, with options for vector alignment using a transformation matrix.
  • USD Property Queries
    • Query objects in a 3D scene using USD properties, such as finding all assets with a specific semantic label.
  • Asset Dependency Tracking
    • Build scene dependency graphs from USD, material, and MDL references. Use cases include: tracking assets impacted by updates, tracking assets reusability, building manifests, and optimizing workflows and scene loading.
  • Combined Query Capabilities
    • Enable complex scenarios for enhanced scene understanding, manipulation, and generation by combining different query types.
• Added USD Search API Python Client library for ease of development - available in Github.
• Added in-scene search into sample kit extension for ease of development.
• Added Terraform module for quickstart with Amazon EKS for ease of development.
• Added indexing status endpoint that allows checking processing state for individual assets.
• Storage backend - added Amazon S3 as official support in addition to Enterprise Nucleus server.

Improved

• Improved Quick startup scripts and general installation documentation for ease of development.
• Improved background indexing speed for performance improvements.

System Requirements

• Kubernetes cluster with the following features enabled:
  • Role-based access control (RBAC) - required for creating GPU-based asset rendering jobs
  • NVIDIA k8s device plugin - required for execution of asset rendering jobs. Alternatively, NVIDIA GPU-operator (which includes NVIDIA k8s device plugin) could be used.
  • Asset rendering jobs require RTX-GPUs to be available on the cluster.
  • Metrics server - required for Horizontal Pod Autoscaling (HPA)
• Hardware
  • Minimum 2 GPUs required (1 for rendering, 1 for inference)
  • CPU: 32 (minimum requirement)
  • RAM: 128 GB (minimum requirement)
  • NVIDIA GPU architecture: Ampere and beyond
• Software:
  • Driver:
    • Recommended: 595.58.03 (GameReady, Studio, RTX/Quadro)
    • Please refer to Omniverse Kit Technical Requirements section for more information.
  • Kubernetes or MicroK8s installed
• Storage backend
  • Amazon S3 or any S3 compatible storage
  • Enterprise Nucleus server
  • Azure Blob storage
  • Google Cloud storage

Get Help

Enterprise Support

Get access to knowledge base articles and support cases or submit a ticket.