Protobuf message definitions for NRE rendering with gRPC.
Overview
NuRec GRPC protos contains protobuf message definitions for NRE rendering with gRPC, as well as scripting required to compile them. For use with the NuRec Container Image to compile
Key Features
- Real-Time Neural Rendering — Serve photorealistic novel-view renders through a built-in gRPC API.
Use Cases
- Autonomous Vehicle Simulation — Convert real-world driving recordings into 3D environments for closed-loop AV testing and validation.
- Robotics (Real-to-Sim) — Create simulation-ready 3D scenes from physical environments to train and evaluate robotic agents.
- Sensor Simulation — Generate synthetic camera and lidar data from reconstructed scenes for sensor model development and testing.
- 3D Asset Generation — Reconstructed scenes can be used to extract individual 3D object assets for populating simulation worlds.
System Requirements
| Component | Requirement |
|---|---|
| GPU | NVIDIA GPU with CUDA compute capability; minimum 24 GB VRAM (48 GB+ recommended) |
| GPU Architectures | Ampere (A100, A10, A40, RTX A6000), Ada Lovelace (L20, L40, L40S), Grace Hopper (H20, H100), Blackwell (RTX Pro 6000D) |
| OS | Linux x86_64 |
| CUDA | 12.8 or higher |
| NVIDIA Driver | R560+ (R570+ recommended; R580+ for Blackwell) |
| Docker | 23.0.1 or later |
| NVIDIA Container Toolkit | 1.13.5 or later |
Supported Input Data
- Camera: images from pinhole, fisheye, and f-theta camera models; rolling and global shutter support; optional windshield distortion modeling.
- Lidar: Structured spinning lidar data (e.g., Hesai Pandar, AT128).
- Pose: Rig-to-world transforms with timestamps.
- Dataset Formats: NCore V3/V4
Data Handling
This package does not handle data directly - it provides definitions for gRPC interactions with the main container image.
Documentation
Comprehensive documentation is available at https://docs.nvidia.com/nurec/
Governing Terms
Your use of the software and materials is governed by the NVIDIA Software License Agreement (found at https://www.nvidia.com/en-us/agreements/enterprise-software/nvidia-software-license-agreement/) and the Product-Specific Terms for NVIDIA Omniverse (found at https://www.nvidia.com/en-us/agreements/enterprise-software/product-specific-terms-for-omniverse/). Additional Information: MIT License.