LLM NIM Overview

Description:

The LLM NIM container is a NIM microservice designed for use with a broad range of LLMs. It includes a production-ready inference runtime with optimized inference engines from NVIDIA and the community— such as NVIDIA TensorRT-LLM, vLLM, and SGLang. When provided an LLM input at container runtime, stored in either Hugging Face or TensorRT-LLM formats, the NIM identifies the model’s format, architecture, and quantization, selects the inference backend, applies pre-configured settings for the LLM and backend, and starts serving the model for inference. For more information, see the NIM documentation and supported model architectures.

The container components are ready for commercial/non-commercial use.

Warning:

NVIDIA cannot guarantee the security of any models hosted on non-NVIDIA systems such as HuggingFace. Malicious or insecure models can result in serious security risks up to and including full remote code execution. We strongly recommend that before attempting to load it, you manually verify the safety of any model not provided by NVIDIA, through such mechanisms as a) ensuring that the model weights are serialized using the safetensors format, b) conducting a manual review of any model or inference code to ensure that it is free of obfuscated or malicious code, and c) validating the signature of the model, if available, to ensure that it comes from a trusted source and has not been modified.

License/Terms of Use:

The NIM container is governed by the NVIDIA Software License Agreement (found at https://www.nvidia.com/en-us/agreements/enterprise-software/nvidia-software-license-agreement/) and the Product-Specific Terms for NVIDIA AI Products (found at https://www.nvidia.com/en-us/agreements/enterprise-software/product-specific-terms-for-ai-products/).

You are responsible for ensuring that your use of NVIDIA AI Foundation Models complies with all applicable laws.

Deployment Geography:

Global

Getting started with NVIDIA NIM

Deploying and integrating NVIDIA NIM is straightforward thanks to our industry standard APIs. Visit the NIM Container LLM page for release documentation, deployment guides and more.

Security Vulnerabilities in Open Source Packages

Please review the Security Scanning (LINK) tab to view the latest security scan results.

For certain open-source vulnerabilities listed in the scan results, NVIDIA provides a response in the form of a Vulnerability Exploitability eXchange (VEX) document. The VEX information can be reviewed and downloaded from the Security Scanning (LINK) tab.

Get Help

Enterprise Support

Get access to knowledge base articles and support cases or submit a ticket.

Container Version(s):

LLM NIM 1.11

Ethical Considerations:

NVIDIA believes Trustworthy AI is a shared responsibility and we have established policies and practices to enable development for a wide array of AI applications. When downloaded or used in accordance with our terms of service, developers should work with their internal developer team to ensure this model meets requirements for the relevant industry and use case and addresses unforeseen product misuse.

Please report security vulnerabilities or NVIDIA AI Concerns here.