NVIDIA NIM Operator

NVIDIA NIM Operator is a Kubernetes Operator that is designed to facilitate the deployment, management, and scaling of NVIDIA NIM microservices on Kubernetes clusters.

NVIDIA NIM microservices deliver AI foundation models as accelerated inference microservices that are portable across data center, workstation, and cloud, accelerating flexible generative AI development, deployment and time to value.

The Operator manages the life cycle of the following microservices and the models they use:

• NVIDIA NIM for LLMs
• NeMo Retriever Text Embedding NIM
• NeMo Retriever Text Reranking NIM

A Helm chart is provided for easily deploying the NIM operator in a cluster to provision the NVIDIA NIMs on GPU-enabled nodes.

Usage

For information on platform support and getting started, visit the official documentation repository

License Agreements

The NVIDIA NIM Operator source code is licensed under Apache 2.0 and contributions are accepted with a DCO. See the contributing document for more information on how to contribute and the release artifacts.

An End User License Agreement is included with this product. By pulling and using the containers from NGC, you accept the terms and conditions of this license.

Suggested Reading

The NIM Operator is open-source. For more information on contributions and release artifacts, see the GitHub repo