Skip to main content
NVIDIA
CDF Augmentation
Container
NVIDIA
CDF Augmentation

CDF Augmentation is an image & video augmentation pipeline that leverages NVIDIA Cosmos and other generative models to produce photorealistic synthetic image and video data. It supports multiple augmentation strategies including weather, lighting, and scene transformations for training and validating models in NVIDIA Metropolis applications.

The Cosmos Data Factory (CDF) Augmentation Pipeline is a containerized generative-AI pipeline for converting ordinary camera data into large, diverse, physically grounded datasets for model training and evaluation. It accepts multimodal inputs—including images, video, and text—and can enrich or transform them through a configurable sequence of preprocessing, captioning, prompt generation, augmentation, verification, and post-processing steps.

Designed for batch-scale, unattended data-generation workflows, the pipeline combines flexible model integration with automated quality controls. Teams can use it to generate broader environmental, visual, and semantic variation from existing data while maintaining repeatability, traceability, and configurable acceptance standards.

Flexible Model Backends

The pipeline supports multiple augmentation backends, including Cosmos Transfer, Cosmos Predict NIM, image-edit models, and image-to-video models, as well as hosted models such as Gemini 3 Pro Image and Veo 3.1. Each model—including the vision-language and language models used for captioning and verification, such as Qwen or Gemma—is accessed through a remote HTTP endpoint and defined as an entry in the configuration file’s endpoints list.

This endpoint-based architecture separates the pipeline from any specific inference runtime or model provider. Model weights are not bundled into the container, which keeps the image lightweight and enables teams to select the models and infrastructure that best fit their requirements. Compatible backends include:

  • OpenAI-compatible inference servers, such as vLLM and vLLM-OMNI
  • NVIDIA Inference Microservices using the /v1/infer contract
  • Hosted model services that expose one of these contracts, such as Gemini 3 Pro Image and Veo 3.1 through NVIDIA-hosted APIs
  • Custom internal models exposed through supported API contracts

As a result, adding, replacing, or retargeting a model typically requires a configuration update rather than a pipeline code change.

Configurable Generation Workflow

The entire workflow is controlled through a single validated configuration file. This makes model selection, endpoint routing, prompt strategy, control weights, generation settings, quality thresholds, and retry behavior declarative and reproducible across runs.

Optional stages can be enabled or adapted to fit a given dataset and generation objective:

  • Preprocessing resizes source images to the input resolution a generation model expects.
  • Captioning derives descriptions or structured context from images and video.
  • Prompt generation creates augmentation instructions from source context. Prompts can also come from fixed text or a prompt file.
  • Template prompts fill a hand-tuned template with a VLM scene description and selected attributes, composing the final prompt deterministically without a second LLM call, for when repeatable control is more important than free-form variation.
  • Generative augmentation applies the selected model backend and generation parameters.
  • Verification evaluates generated outputs against configured quality and attribute requirements.
  • Post-processing aligns outputs back to the source frame and normalizes generated video to a single codec (VP9).

For captioning and attribute verification, the pipeline can connect to any OpenAI-compatible language or vision-language model, whether served locally or accessed from a remote provider. The example configurations use Qwen-family models, while teams remain free to use newer or domain-specific models as their needs evolve.

Automated Quality Assurance

Built-in quality gates help ensure that generated data is suitable for downstream use. The pipeline can detect undesirable motion artifacts that were not present in the source material, use AI-generated multiple-choice verification to assess required output attributes, and delegate additional checks to an external Cosmos Evaluator service.

When evaluators are configured, each generated result is checked against them automatically. If an output fails a check, the pipeline can regenerate it with an incremented seed (and, by default, a refreshed prompt), repeating the process up to a configurable retry limit. Outputs that still fail after all retry attempts are marked as failed rather than silently accepted, and the run exits with a non-zero status.

This approach provides a practical balance between automated throughput and dataset reliability:

  • Quality criteria remain explicit and configurable.
  • Regeneration is automatic rather than dependent on manual intervention.
  • Retry limits prevent uncontrolled compute use.
  • Failed outputs are retained by default, together with their evaluation results, for review, auditing, or later processing.

Deployment and Operations

The Augmentation Pipeline is distributed as a published CUDA-based container and is operated from the command line rather than as a persistent network service. This model makes it well suited for backend automation, scheduled jobs, CI/CD-style data workflows, and custom generation pipelines.

Users run the container and supply a configuration file that defines the workflow. Because all model inference happens on remote endpoints, the container itself does not need a GPU for inference. A GPU is required only for local media processing: decoding H.264 video, for which the image ships only the NVDEC hardware decoder, and the optional alignment post-processor. Command-line overrides can be used to change inputs, select different models, or adjust parameters for a specific run without modifying the base configuration.

The pipeline is also aware of NVIDIA Cloud Functions task environments. When executed within a supported cloud-task context, it automatically detects the environment and reports incremental progress, allowing external orchestration systems to monitor and manage longer-running generation jobs.

Storage and Data Movement

Inputs and outputs are handled through a flexible storage layer that supports both local and cloud-based data sources. The pipeline can operate with:

  • Local filesystem paths
  • Amazon S3 and S3-compatible object storage
  • Google Cloud Storage through its S3-compatible interface
  • Azure object storage
  • HTTPS object URLs mapped to a configured storage profile

This allows the same augmentation workflow to run across local development environments, on-premises GPU clusters, and managed cloud infrastructure without changing the pipeline’s core behavior.

Governing Terms

Our containers are governed by the NVIDIA Software License Agreement and Product-Specific Terms for NVIDIA AI Products.

Publisher
NVIDIA
Latest Tag1.2.0
UpdatedOctober 19, 2026 UTC
Compressed Size4.63 GB
Multinode SupportNo
Multi-Arch SupportYes

Modal Content

NVIDIA uses cookies to improve your experience on our web site. We and our third-party partners also use cookies and other tools to collect and record information you provide as well as information about your interactions with our websites for performance improvement, analytics, and to assist in marketing efforts. By clicking "Accept All", you consent to our use of cookies and other tools as described in our Cookie Policy. You can manage your cookie settings by clicking on "Manage Settings." By continuing to use this site or by clicking one of the buttons below, you agree to our Terms of Service (which contains important waivers). Please see our Privacy Policy for more information on our privacy practices.