Step-5 Preview Serving Recipes

NIM tuning config files for Step-5 Preview on nvcr.io/nim/nvidia/vllm-model-free-nim-experimental:2.1.6-variant (NIM_STACK_COMMIT=036eb9f68a8285da140b2075733d8e180956bb99). Each file covers one GPU and parallelism combination:

The header comment in each file names the source recipe and the GPU memory range and checkpoint it was validated for. Every file serves the model as stepfun-ai/Step-5-Preview; use this name as the model field in API requests.

How NIM applies the recipes

This container build does not select Step-5 Preview recipes automatically. To use a validated configuration, load the matching recipe file as the NIM tuning config.

Explicit tuning config file. Set NIM_CONFIG_FILE to one of the recipe files to load it as the NIM tuning config. Top-level keys are vLLM settings, and the ENV section sets environment variables. Without a tuning config file, NIM starts Step-5 Preview with its generic defaults, which are outside the validated configuration.

Load a recipe file

Download the recipes, pick the file for your GPU, and make it readable inside the container:

ngc registry resource download-version "nim/nvidia/step5-preview-recipe:2.1.6-variant"
export RECIPE="$PWD/step5-preview-recipe_v2.1.6-variant/step5-preview-nvfp4-gb200-4gpu-tp4-dp1-ep-mtp3.yaml"
chmod 0644 "$RECIPE"

NGC CLI saves downloaded files with mode 0600, owned by the host user. The container runs as UID 1000, so it can't read a 0600 file that a different host user owns, and NIM fails to load NIM_CONFIG_FILE with Permission denied. The chmod changes only the file mode, not its contents.

Pick the Hugging Face checkpoint that matches the recipe file:

Mount the recipe file into the container, point NIM_CONFIG_FILE at it, and set NIM_MODEL_PATH to the checkpoint:

export IMAGE=nvcr.io/nim/nvidia/vllm-model-free-nim-experimental:2.1.6-variant
export NIM_MODEL_PATH=hf://stepfun-ai/Step-5-Preview-NVFP4
export HF_TOKEN=<your-hugging-face-token>

docker run --rm \
  --gpus 4 \
  --ipc=host \
  --ulimit memlock=-1 \
  --ulimit stack=67108864 \
  -p 8000:8000 \
  -v step5-nim-cache:/opt/nim/.cache \
  -v "$RECIPE:/opt/nim/step5-recipe.yaml:ro" \
  -e NIM_MODEL_PATH \
  -e HF_TOKEN \
  -e NIM_CONFIG_FILE=/opt/nim/step5-recipe.yaml \
  "$IMAGE" \
  nim-serve

Use --gpus 8 with the H20, H20-3e, and H200 files. On first start NIM downloads the checkpoint into the step5-nim-cache volume, so the volume needs room for the full checkpoint; later starts reuse the cached copy. Both repositories are public, and HF_TOKEN avoids anonymous download rate limits.

Override a setting

Settings are resolved in this order, from lowest to highest priority: tuning config file, environment variables, NIM_PASSTHROUGH_ARGS, and nim-serve command-line options. To change one value, pass it on the command line, for example nim-serve --max-num-seqs 128. Changing recipe settings moves the deployment outside the validated configuration.

Verify the service

curl --fail http://localhost:8000/v1/health/ready
curl --fail http://localhost:8000/v1/models

Get Help

NVIDIA Developer Community Forum

Get access to community knowledge base articles and support cases (https://forums.developer.nvidia.com/)