Semantic Split C-RADIO NIM for content-aware semantic video splitting of raw long-form videos.
Semantic Split C-RADIO NIM
Overview
Semantic Split C-RADIO is an inference system for content-aware semantic video splitting of raw long-form videos. NVIDIA C-RADIO model generates image embeddings required within the stitching stage.
Semantic Split C-RADIO NIM is designed to process one or more videos and extract scenes of interest, providing semantically distinct data as video clips. The NIM provides GPU-accelerated capabilities to detect scene transitions and filter out still and duplicate scenes. It can decode various input formats and re-encode them into an homogeneous format.
The Semantic Split C-RADIO NIM follows the algorithmic design from the Panda-70M paper. It leverages:
- A GPU-accelerated version of PySceneDetect for initial detection of scenes (splitting)
- The commercial RADIO model for semantic understanding (frame embeddings), to perform scene stitching and filtering (filtering of scene transitions, still and redundant scenes)
- PyNvVideoCodec for accelerated decoding and encoding capabilities and CV-CUDA for video pre-processing
- FFMPEG to read the video metadata, perform audio muxing and file splitting (no decoding/encoding)
NVIDIA NIM
NVIDIA NIM, part of NVIDIA AI Enterprise, is a set of easy-to-use microservices designed to speed up generative AI deployment in enterprises. Supporting a wide range of AI models, including NVIDIA AI foundation and custom models, it ensures seamless, scalable AI inferencing, on-premises or in the cloud, leveraging industry standard APIs. Learn more abot NVIDIA NIM here.
Intended Use
Video Curation/Video Processing: Semantic splitting is an essential stage in the video curation workflow for preparing training data for video/multi-modal models.
Please review the NIM evaluation EULA before use.
Getting Started with NIM
Refer to Semantic Split C-RADIO User Guide for more details on the NIM architecture, prerequsities, getting started, performance, troubleshooting, and more. For EA customers, the user guide (PDF) is provided directly to approved EA applicants.
NIM Details
Supported Format
Input
- Input codecs: H.264, HEVC, VP9, VP8, AV1
- Input pixel formats: NV12/YUV420p
- Input container types: MP4, MOV, WEBM
- Maximum input resolution: 1080p
Output
- Output codec and container: H.264, MP4
- Output resolution: Same as input
Hardware
Tested and qualified on NVIDIA L40S.
Model Details
C-RADIO is a commercial-ready version of NVIDIA's RADIO model. For more details on RADIO, refer here.
References
Terms of Use (Early Access)
By using the NVIDIA software, models or documentation you agree to fully comply with the terms and conditions of NVIDIA Software and Model Evaluation License. The EULA PDF is provided directly to approved EA applicants.
If you do not agree to these terms and conditions, do not install, download or use the NVIDIA software, models or documentation.
Governing Terms
This Early Access NIM container and model is governed by NVIDIA Software and Model Evaluation License Agreement.
Ethical Considerations:
NVIDIA believes Trustworthy AI is a shared responsibility and we have established policies and practices to enable development for a wide array of AI applications. When downloaded or used in accordance with our terms of service, developers should work with their internal model team to ensure this model meets requirements for the relevant industry and use case and addresses unforeseen product misuse.
Please report security vulnerabilities or NVIDIA AI Concerns here.
Get Help
NVIDIA Developer Community Forum
For support, Visit the NVIDIA Developer Community Forum
End of Support — "This artifact is no longer supported. NVIDIA strongly recommends artifacts that are up to date and supported"