ContainersNVIDIADeepSeek-R1NVIDIALlama-3.1-Nemotron-70B-InstructNVIDIAPyTorch
CollectionsNVIDIAOmniverse Kit (FB)NVIDIADeepStream SDKNVIDIAOmniverse Kit App Streaming
ModelsNVIDIAStyleGAN3 pretrained modelsNVIDIAPeopleNetNVIDIATrafficCamNet
ResourcesNVIDIARiva Skills Quick Start
Helm ChartsNVIDIAGPU OperatorNVIDIANVIDIA NIM Operator
  • NVIDIA
    Cosmos-Embed1
    Model
    NVIDIA
    Cosmos-Embed1

    Cosmos-Embed1 is a joint video-text embedder tailored for physical AI.

  • This model is backed by NVIDIA's Plus Plus (++) Promise
    to learn more about the quality of the datasets used to train this model.
    FieldResponse
    Intended Task/Domain:Joint video-text embedding for physical AI applications (robotics, autonomous vehicles (AV), search, general video understanding) and video anomaly detection and classification.
    Model Type:Transformer
    Intended Users:Physical AI developers working on robotics, autonomous vehicles (AV), search, video understanding, and video anomaly detection tasks.
    Output:L2-normalized text and video embedding vectors (256 dimensions for 224p inputs; 768 dimensions for 336p/448p inputs).
    Describe how the model works:Video frames are individually processed by a ViT backbone, temporally augmented, and compressed via QFormer cross-attention into a single video embedding. Text is processed by the QFormer self-attention branch into a text embedding. Both embeddings are aligned via contrastive learning.
    Name the adversely impacted groups this has been tested to deliver comparable outcomes regardless of:Not Applicable
    Technical Limitations & Mitigation:Due to the training datasets being predominantly composed of short action-focused English captions, different text prompts may not be properly aligned to video data, producing suboptimal matching. The model has been optimized for 8 frames at 1–2 FPS; significantly different frame rates or video lengths may degrade performance. Fine-tuning on domain-specific data can mitigate these limitations.
    Verified to have met prescribed NVIDIA quality standards:Yes
    Performance Metrics:Retrieval metrics (top-K hit rate, mean reciprocal rank) and classification metrics (precision, recall, F1)
    Potential Known Risks:The model may produce suboptimal embeddings to unsafe text prompts.
    Licensing:GOVERNING TERMS: Use of this model is governed by the NVIDIA Open Model License. Additional Information: Apache 2.0 and MIT.

    NVIDIA uses cookies to improve your experience on our web site. We and our third-party partners also use cookies and other tools to collect and record information you provide as well as information about your interactions with our websites for performance improvement, analytics, and to assist in marketing efforts. By clicking "Accept All", you consent to our use of cookies and other tools as described in our Cookie Policy. You can manage your cookie settings by clicking on "Manage Settings." By continuing to use this site or by clicking one of the buttons below, you agree to our Terms of Service (which contains important waivers). Please see our Privacy Policy for more information on our privacy practices.