PAIDF Augmentation is an image & video augmentation pipeline that leverages NVIDIA Cosmos and other generative models to produce photorealistic synthetic video data. It supports multiple augmentation strategies including weather, lighting, and scene transformations for training and validating models in NVIDIA Metropolis applications.
The Physical AI Data Factory (PAIDF) Augmentation Pipeline is a containerized generative-AI service that transforms ordinary camera data into large, diverse, physically grounded datasets for training and evaluation. It accepts a range of inputs—video, single images, or text—together with optional structural guidance such as edge, depth, and segmentation maps, and routes each request to the appropriate generative model. Every run produces a complete, machine-readable bundle: the augmented media, the exact generation prompt, and a metadata record that captures the prompt, the sampled target attributes, the input and output locations, and the results of each quality check. Built-in quality gates flag motion artifacts absent from the source, plus AI-generated multiple-choice attribute verification—automatically score every output and, when a result falls short, regenerate it with a new seed up to a configurable retry limit before accepting or flagging it. This makes the pipeline well suited to unattended, batch-scale data factories that demand consistent, trustworthy results.
A defining strength of the Augmentation Pipeline is its model flexibility and configuration-driven customization. It selects among three generative backends to match each job: Cosmos Transfer 2.5 for video-to-video style transfer, Cosmos Predict 2.5 for text-, image-, or video-conditioned video generation, and an image-editing model for single-image edits. For captioning and attribute verification, the pipeline connects to any OpenAI-compatible model—served locally or by a remote provider—with Qwen-family models configured as sensible defaults, ensuring users can always leverage the latest models for their specific needs. Prompt creation is equally adaptable, letting teams choose between AI-generated, context-aware prompts and deterministic, hand-tuned ones with no code changes. All of this behavior is expressed in a single, validated configuration file, so model selection, control weights, generation parameters, and quality thresholds remain fully declarative and reproducible across runs.
The Augmentation Pipeline is delivered as a published GPU container and is driven from the command line rather than as a long-running network service, making it ideal for backend automation and custom data-generation workflows. Users launch the container on an NVIDIA GPU and run the pipeline against a configuration file, with simple command-line overrides available to retarget inputs, swap models, or tune parameters inline. Generation runs locally across multiple GPUs with no remote endpoint required, while alternate execution modes can offload generation to remote services when desired. To scale beyond single runs, a workflow generator samples attribute distributions—including dependent, conditional attributes—to emit hundreds of per-sample configurations, and a companion dataset builder assembles image-editing outputs into a structured, query-annotated dataset. The pipeline is also cloud-task aware: when run inside an NVIDIA Cloud Functions task environment it automatically detects the context and reports incremental progress for orchestration within larger managed jobs, while all inputs and outputs flow through a flexible storage layer that spans the local filesystem and cloud object storage such as S3, GCS, Azure, or HTTP-backed storage.
Governing terms
Our containers are governed by the NVIDIA Software License Agreement and Product-Specific Terms for NVIDIA AI Products.