Model Card - Fast-FoundationStereo

Model Overview

Description:

Fast-FoundationStereo estimates the disparity of each pixel in a rectified binocular stereo pair of images. It is a transformer-based foundational model that shows strong zero-shot generalization while running in real time. Fast-FoundationStereo was developed by NVIDIA. This model is ready for commercial or non-commercial use.

License/Terms of Use:

This model is released under the NVIDIA Open Model Agreement: https://www.nvidia.com/en-us/agreements/enterprise-software/nvidia-open-model-agreement/

Deployment Geography:

Global

Use Case:

Researchers and developers in the field of computer vision, specifically those interested in depth estimation, are expected to use this method for tasks such as three-dimensional reconstruction, object detection, object pose estimation, and scene understanding.

Release Date:

Github 02/01/2026 via https://github.com/NVlabs/Fast-FoundationStereo

References(s):

Fast-FoundationStereo: Real-Time Zero-Shot Stereo Matching

Model Architecture:

Architecture Type: Transformer and Convolutional Neural Network (CNN)

Network Architecture: The network contains three parts: 1) an EdgeNeXt student module that distills the original FoundationStereo feature extractor; 2) a set of CNN and transformer blocks that perform matching with long-range dependencies; 3) a reduced set of convGRU blocks.

This model was developed based on FoundationStereo.

Number of model parameters: 14.6M (1.46*10^7).

Computational Load (Internal Only: For NVIDIA Models Only)

Cumulative Compute: Trained on 32 × NVIDIA A100-SXM4-80GB GPUs (4 nodes × 8 GPUs) for 206,389 s of wall-clock time (~57.3 h, ~1.83 × 10^3 GPU-hours) using FP16 mixed-precision. At the A100 dense FP16 Tensor Core peak of 312 TFLOPS, this corresponds to an upper-bound cumulative compute of ~2.06 × 10^21 FLOPs.
Estimated Energy and Emissions for Model Training: Using the A100-SXM4-80GB TDP of 400 W, GPU energy ≈ 734 kWh. Applying a 1.5× node-level overhead (CPU, DRAM, NIC, storage; Patterson et al. 2021) and a 1.2 data-center PUE yields ≈ 1.32 × 10^3 kWh of total system energy. Assuming a grid carbon intensity of 0.37 kg CO₂eq/kWh (U.S. average), estimated emissions ≈ 4.9 × 10^2 kg CO₂eq.

Input(s):

Input Type(s): Image (a pair of two rectified binocular stereo images)

Input Format(s):

• Image: Red, Green, Blue (RGB)
  

Input Parameters:

• Image: Two-Dimensional (2D)
  

Other Properties Related to Input: Rectified stereo image pair from a camera such as Zed. The baseline is needed to convert disparity to depth. No alpha channel and no pre-processing needed. Bit depth: 24-bit.

Output(s)

Output Type(s): Image (disparity map)

Output Format(s):

• Image: 16-bit unsigned integer
  

Output Parameters:

• Image: Two-Dimensional (2D)
  

Other Properties Related to Output: Final 2D disparity map. No alpha channel and no post-processing needed. Bit depth: 16-bit.

Our AI models are designed and/or optimized to run on NVIDIA GPU-accelerated systems. By leveraging NVIDIA's hardware (e.g. GPU cores) and software frameworks (e.g., CUDA libraries), the model achieves faster training and inference times compared to CPU-only solutions.

Software Integration:

Runtime Engine(s):

• Not Applicable (N/A)
  

Supported Hardware Microarchitecture Compatibility:

• NVIDIA Ampere
  

Supported Operating System(s):

• Linux
  

The integration of foundation and fine-tuned models into AI systems requires additional testing using use-case-specific data to ensure safe and effective deployment. Following the V-model methodology, iterative testing and validation at both unit and system levels are essential to mitigate risks, meet technical and functional requirements, and ensure compliance with safety and ethical standards before deployment.

Model Version(s):

v1.0: Initial model version with full capabilities, unpruned and trained.

Training, Testing, and Evaluation Datasets:

Training Dataset:

Data Modality:

• Image
  

Training Data Size:

** Image Training Data Size

• 1 Million to 1 Billion Images
  

** Data Collection Method by dataset

• Hybrid: Synthetic, Automatic/Sensors
  

** Labeling Method by dataset

• Hybrid: Synthetic, Automatic/Sensors
  

Properties: The training dataset includes: 1) a large-scale internal proprietary synthetic dataset featuring 1.4 million stereo pairs with a large diversity of objects and scenes and high photorealism; 2) the external commercial SDG.

Link: Internal, proprietary dataset, and external commercial SDG.

Testing Dataset:

Link: Middlebury dataset.

Data Collection Method by dataset:

• Automatic/Sensors
  

Labeling Method by dataset:

• Automatic/Sensors
  

Properties: The dataset encompasses a wide range of scenarios, includes diverse three-dimensional assets, captures stereo images under diversely randomized camera parameters, and achieves high fidelity in both rendering and spatial layouts.

Evaluation Dataset:

Benchmark Score: Evaluated on the public leaderboards well known to the stereo community.

• Middlebury: https://vision.middlebury.edu/stereo/ — the classic leaderboard for dense stereo matching, developed at Middlebury College.
  
• ETH3D: https://www.eth3d.net/ — a popular leaderboard for stereo, developed at ETH.
  
• KITTI: https://www.cvlibs.net/datasets/kitti/eval_stereo.php — a cornerstone benchmark for autonomous driving research.
  

Data Collection Method by dataset:

• Automatic/Sensors
  

Labeling Method by dataset:

• Automatic/Sensors
  

Properties:

• The Middlebury Stereo dataset consists of high-resolution stereo sequences with complex geometry and pixel-accurate ground-truth disparity data. The ground-truth disparities were acquired using a novel technique that employs structured lighting and infrared paint.
  
• ETH3D is a multi-view stereo / 3D reconstruction benchmark that covers a variety of indoor and outdoor scenes. Ground-truth geometry was obtained using a high-precision laser scanner. A DSLR camera as well as a synchronized multi-camera rig with varying field-of-view was used to capture images.
  
• KITTI stereo dataset is a cornerstone of autonomous driving research, developed by the Karlsruhe Institute of Technology (KIT) and the Toyota Technological Institute at Chicago (TTIC). It provides real-world, high-resolution stereo imagery paired with precise ground-truth depth data collected from a moving vehicle in diverse urban environments.
  

Inference:

Acceleration Engine: TensorRT
Test Hardware:

• Zed Stereo Camera
  
• NVIDIA GeForce RTX 3090
  

Ethical Considerations:

NVIDIA believes Trustworthy AI is a shared responsibility and we have established policies and practices to enable development for a wide array of AI applications. When downloaded or used in accordance with our terms of service, developers should work with their internal model team to ensure this model meets requirements for the relevant industry and use case and addresses unforeseen product misuse.

Please make sure you have proper rights and permissions for all input image and video content; if image or video includes people, personal health information, or intellectual property, the image or video generated will not blur or maintain proportions of image subjects included.

Please report model quality, risk, security vulnerabilities or NVIDIA AI Concerns here.