NeMo Microservices Platform Helm Chart

The NVIDIA NeMo Microservices Platform Helm chart simplifies the deployment process by bundling all individual Helm charts of the NeMo microservices into one easy-to-use parent chart. This streamlines deployment while providing flexibility in configuration to meet your specific requirements.

Included Microservices

The platform includes multiple microservices working together.

Core Microservices

• NVIDIA NeMo Customizer: Facilitates the fine-tuning of large language models (LLMs) using supervised and parameter-efficient fine-tuning techniques.
• NVIDIA NeMo Evaluator: Provides comprehensive evaluation capabilities for LLMs, supporting academic benchmarks, custom automated evaluations, and LLM-as-a-Judge approaches.
• NVIDIA NeMo Guardrails: Adds safety checks and content moderation to LLM endpoints, protecting against hallucinations, harmful content, and security vulnerabilities.

Platform Component Microservices

• NVIDIA NeMo Data Store: Serves as the default file storage solution for the NeMo microservices platform, exposing APIs compatible with the Hugging Face Hub client (HfApi).
• NVIDIA NeMo Entity Store: Provides tools to manage and organize general entities such as namespaces, projects, datasets, and models.
• NVIDIA NeMo Deployment Management: Provides an API to deploy NIM for LLMs on a Kubernetes cluster and manage them through the NIM Operator microservice.
• NVIDIA NeMo NIM Proxy: Provides a unified endpoint that you can use to access all deployed NIM for LLMs for inference tasks.
• NVIDIA NeMo Operator: Manages custom resource definitions (CRDs) for NeMo Customizer fine-tuning jobs.
• DGX Cloud Admission Controller: Enables multi-node training requirements for NeMo Customizer jobs through a mutating admission webhook.

Resources

Chart | Helm Installation Guide

Note: Use, distribution or deployment of these microservices in production requires an NVIDIA AI Enterprise License.

Governing Terms

The software and materials are governed by the NVIDIA Software License Agreement and the Product-Specific Terms for NVIDIA AI Products.