This collection houses SGLang Model-Free NIM and model-specific deployment recipes.
SGLang Model-Free NIM Experimental Collection
Description
This collection provides the assets needed to deploy a model as a model-free NIM: a generic NIM container that serves a model supplied at deploy time, and validated, model-specific recipes that carry the launch settings for each supported GPU and topology.
Model-Free NIM: unlike a model-specific NIM, the model-free NIM ships no baked-in model weights. The model is selected at deploy time through the NIM_MODEL_PATH environment variable, so a single image can serve different models and revisions without a rebuild.
Recipe: because the container is shared across models, it cannot include settings that only one model needs. Recipes fill that gap. Each recipe is a YAML file, validated by NVIDIA for one model on one GPU type and topology, that pins the engine arguments and environment variables. The NIM plans and starts the engine with those settings.
To deploy, run the container with the model mounted through NIM_MODEL_PATH and a recipe passed through NIM_CONFIG_FILE. Each recipe version matches the container tag it was validated with; the compatibility matrix in this collection lists every validated pairing of container, recipe and model.
| Container version | Recipe version | Model |
|---|---|---|
| 2.1.7-variant | 2.1.7-variant | hf://MiniMaxAI/MiniMax-M3.1-Flash |
Collection Contents
| Asset | Type | Description |
|---|---|---|
| SGLang Model-Free NIM Experimental | Container | The NIM, built on SGLang. Serves any supported model; the model is supplied at deploy time through NIM_MODEL_PATH. Supported profiles, environment variables, and security scan results are on the container card. |
| Model-Specific Recipes | Resource | Validated recipes for supported models, GPU SKUs, and topologies. Each YAML file pins engine arguments, and environment variables. Versions match the container tag they were validated with. |
Deployment Details
Deploying from this collection requires Docker with the NVIDIA Container Toolkit and access to nvcr.io (docker login nvcr.io with your NGC API key). Download the model weights and the recipe matching your model, GPU SKU, and topology from the Resources section, using the recipe version that matches your container tag. Then start the container with docker run, mounting the model through NIM_MODEL_PATH and passing the recipe file through NIM_CONFIG_FILE.
Step 1: Download recipe and pick one that matches your GPU.
ngc registry resource download-version "nim/nvidia/<recipe-resource>:<version>"
export RECIPE="$PWD/<recipe-resource>_v<version>/<recipe-file>.yaml"
Step 2: Mount the file into container and deploy
export IMAGE=nvcr.io/nim/nvidia/sglang-model-free-nim-experimental:<version>
export NIM_MODEL_PATH="hf://MiniMaxAI/MiniMax-M3.1-Flash"
export HF_TOKEN="hf_xxx"
export LOCAL_NIM_CACHE=~/.cache/nim
mkdir -p "$LOCAL_NIM_CACHE"
chmod -R a+rwxt "$LOCAL_NIM_CACHE"
docker run --gpus='"device=0,1,2,3"' \
-e NIM_MODEL_PATH=$NIM_MODEL_PATH \
-e HF_TOKEN=$HF_TOKEN \
-v "$LOCAL_NIM_CACHE:/opt/nim/.cache" \
-v "$RECIPE:/opt/nim/recipe.yaml:ro" \
-e NIM_CONFIG_FILE="/opt/nim/recipe.yaml" \
-p 8000:8000 \
$IMAGE
License/Terms of Use
GOVERNING TERMS: The NIM container is governed by the NVIDIA Software License Agreement and the Product-Specific Terms for NVIDIA AI Products.
The use of any models with this container is governed by the NVIDIA Open Model Agreement in addition to any terms which govern the specific models used.
You are responsible for ensuring that your use of any provided models complies with all applicable laws.
Models provided in this collection are Nemotron models, whose use is governed by the NVIDIA Nemotron Open Model License.
Deployment Geography
Global
Security Common Vulnerabilities and Exposures (CVEs)
Please review the Security Scanning tab on the container within this collection to view the latest security scan results. For certain open-source vulnerabilities listed in the scan results, NVIDIA provides a response in the form of a Vulnerability Exploitability eXchange (VEX) document. The VEX information can be reviewed and downloaded from the Security Scanning tab.
Ethical Considerations
NVIDIA believes Trustworthy AI is a shared responsibility and we have established policies and practices to enable development for a wide array of AI applications. When downloaded or used in accordance with our terms of service, developers should work with their internal teams to ensure these software components meet requirements for the relevant industry and use case and address unforeseen product misuse.
Please report quality , risk , security vulnerabilities or NVIDIA AI Concerns here.
Get Help
Getting started with the NIM
Deploying and integrating the NIM is straightforward thanks to our industry standard APIs. Visit the NIM Container page for release documentation, deployment guides and more NIM Container LLM.
Enterprise Support
Get access to knowledge base articles and support cases or submit a ticket.
