NeMo Microservices Helm Chart

NVIDIA NeMo Microservices Helm Chart simplifies the deployment process by bundling all individual Helm charts of the NeMo microservices into one easy-to-use parent chart. This streamlines deployment while providing flexibility in configuration to meet your specific requirements.

Included Microservices

This chart includes the following microservices working together.

Functional Microservices

The following are the functional microservices that are the main components of the NeMo microservices platform, forming a complete data flywheel.

• NVIDIA NeMo Customizer: Facilitates the fine-tuning of large language models (LLMs) using supervised and parameter-efficient fine-tuning techniques.
• NVIDIA NeMo Evaluator: Provides comprehensive evaluation capabilities for LLMs, supporting academic benchmarks, custom automated evaluations, and LLM-as-a-Judge approaches.
• NVIDIA NeMo Guardrails: Adds safety checks and content moderation to LLM endpoints, protecting against hallucinations, harmful content, and security vulnerabilities.
• NVIDIA NeMo Data Designer: Generates high-quality synthetic datasets using AI models, statistical sampling, and configurable data schemas.
• NVIDIA NeMo Auditor (Early Access): Audits models and agentic applications for security vulnerabilities and harmful content.

Infrastructure Microservices

The following are the infrastructure microservices that form the infrastructure for the functional microservices.

• NVIDIA NeMo Data Store: Serves as the default file storage solution for the NeMo microservices platform, exposing APIs compatible with the Hugging Face Hub client (HfApi).
• NVIDIA NeMo Entity Store: Provides tools to manage and organize general entities such as namespaces, projects, datasets, and models.
• NVIDIA NeMo Deployment Management: Provides an API to deploy NIM for LLMs on a Kubernetes cluster and manage them through the NIM Operator microservice.
• NVIDIA NeMo NIM Proxy: Provides a unified endpoint that you can use to access all deployed NIM for LLMs for inference tasks.
• NVIDIA NeMo Operator: Manages custom resource definitions (CRDs) for NeMo Customizer fine-tuning jobs.
• DGX Cloud Admission Controller: Enables multi-node training requirements for NeMo Customizer jobs through a mutating admission webhook.

Resources

Collection | Helm Installation Guide | NeMo Microservices Documentation

Note: Use, distribution or deployment of these microservices in production requires an NVIDIA AI Enterprise License.

Get Help

Enterprise Support

Get access to knowledge base articles and support cases or submit a ticket.

Governing Terms

The software and materials are governed by the NVIDIA Software License Agreement and the Product-Specific Terms for NVIDIA AI Products.