LVS uses VLMs to extract insights and summarize long videos.
Long Video Summarization (LVS) Microservice Overview
Long Video Summarization (LVS) is a comprehensive microservice that leverages Vision-Language Models (VLMs) to extract insights from video content. The system generates structured, machine-readable summaries in JSON format, providing timestamped events that pinpoint precisely when and where specific actions, object appearances, or scene changes occurred. Beyond detailed event lists, the service offers high-level narrative summaries of the video's main points, making it ideal for efficiently processing and distilling insights from extensive video content.
A defining feature of LVS is its high degree of customization and model flexibility. The service supports any OpenAI-compatible Vision-Language Model (VLM) or Large Language Model (LLM), ensuring users can leverage the latest models for their specific needs. For scenarios demanding faster inference, LVS offers pre-configured support for highly optimized VLMs such as CR1, CR2, and Qwen. All processed summaries, events, and metadata are stored in a configurable database layer, with Elasticsearch as the default option for its powerful search and analytical capabilities.
The LVS microservice provides multiple integration methods to ensure broad applicability. It exposes a standard RESTful API for programmatic access and traditional software integrations, making it ideal for backend systems and custom applications. Additionally, it offers a Model Context Protocol (MCP) interface specifically designed for communication with AI agents and complex orchestration systems, enabling dynamic and intelligent control over the summarization process within larger AI ecosystems.
Governing terms
Governing Terms: The blueprint deployment files and scripts are governed by the Apache 2.0 License. The blueprint microservices are governed by the NVIDIA Software and Model Evaluation License Agreement. This blueprint software enables use of separate open source and proprietary software, models and datasets governed by their respective licenses below.
- Models, which are governed by the NVIDIA Open Model License
- RT-DETR 2D
- Cosmos-Reason2-2B
- Cosmos-Reason2-8B
- Cosmos-Embed1
- NVIDIA-Nemotron-Nano-9B-v2
- Grounding DINO
- TrafficCamNet Transformer Lite
- Sparse4D 2.1
- RADIO-CLIP v1
- SigLIP v2
- Cosmos-Embed
- MaskGroundingDINO v2.1
- NIMs, which are governed by the NVIDIA Software License Agreement and Product-Specific Terms for NVIDIA AI Products
- Elastic Container [docker.elastic.co/elasticsearch/elasticsearch:9.3.0], which is governed by the Elastic License
- Sample data and assets, which are governed by the NVIDIA Sample Data License for Evaluation