DeepSeek-V4.1-Flash-1017 Recipes

Description

Validated, ready-to-apply launch configurations ("recipes") for serving deepseek-ai/DeepSeek-V4.1-Flash-1017 with the SGLang Model-Free NIM (Experimental) container. Each recipe is a YAML file for one GPU type and topology. Pass it to the container with NIM_CONFIG_FILE, and the container starts with the exact settings NVIDIA validated.

Recipe file names follow {model}-{precision}-{gpu}-{N}gpu-tp{N}-ep{N}-dspark.yaml; the GPU count in the name is the number of GPUs the recipe was validated with. Select the recipe that matches your deployment environment.

The header comment in each file names the checkpoint revision, GPU, GPU count and the container tag it was validated with. Every file serves the model as deepseek-ai/DeepSeek-V4.1-Flash-1017; use this name as the model field in API requests.

Version

Recipe's version equals the container tag it was validated with. Recipe 2.1.6-variant-deepseek-v4.1-flash-1017 is for container sglang-model-free-nim-experimental:2.1.6-variant-deepseek-v4.1-flash-1017.

Do NOT mix versions. A recipe may set engine options that another container version does not have, or that behave differently.

Deployment Details

Step 1 — Download the recipes and pick the file for your GPU.

ngc registry resource download-version "nim/nvidia/deepseek-v4.1-flash-1017-recipe:2.1.6-variant-deepseek-v4.1-flash-1017"
chmod 0644 deepseek-v4.1-flash-1017-recipe_v2.1.6-variant-deepseek-v4.1-flash-1017/*.yaml
export RECIPES="$PWD/deepseek-v4.1-flash-1017-recipe_v2.1.6-variant-deepseek-v4.1-flash-1017"

NGC CLI saves downloaded files with mode 0600. The container runs as UID 1000 and cannot read a 0600 file owned by another host user; chmod changes only the mode.

Step 2 — Provide the weights. Either a local Hugging Face-layout directory (config.json, tokenizer files, 48 safetensors shards, index; about 476 GiB) mounted read-only, or let NIM download from Hugging Face with NIM_MODEL_PATH=hf://deepseek-ai/DeepSeek-V4.1-Flash-1017 (set HF_TOKEN to avoid anonymous rate limits; the cache volume needs room for the full checkpoint).

Step 3 — Start the container with exactly the recipe's GPU count.

export IMAGE=nvcr.io/nim/nvidia/sglang-model-free-nim-experimental:2.1.6-variant-deepseek-v4.1-flash-1017
export MODEL_DIR=/path/to/DeepSeek-V4.1-Flash-1017      # HF layout
export LOCAL_NIM_CACHE=~/.cache/nim && mkdir -p "$LOCAL_NIM_CACHE" && chmod 0777 "$LOCAL_NIM_CACHE"

docker run -d --name dsv41-flash-1017 \
  --gpus '"device=0,1,2,3"' \
  --ipc=host --shm-size=32g \
  -p 8000:8000 \
  -v "$LOCAL_NIM_CACHE:/opt/nim/.cache" \
  -v "$MODEL_DIR:/models/model:ro" \
  -v "$RECIPES:/opt/nim/recipes:ro" \
  -e NIM_MODEL_PATH=/models/model \
  -e NIM_CONFIG_FILE=/opt/nim/recipes/deepseek-v4.1-flash-1017-fp8-b200-4gpu-tp4-ep4-dspark.yaml \
  -e NIM_SERVED_MODEL_NAME=deepseek-ai/DeepSeek-V4.1-Flash-1017 \
  "$IMAGE"

• GPU count must equal the recipe's tp_size: --gpus '"device=0,1,2,3"' (any four GPUs) with the B200/GB200 files, --gpus all on an 8-GPU H200 or H20-3e host with the …-h200-8gpu-… / …-h20-3e-8gpu-… files. If more GPUs are visible than the recipe's tp_size (for example --gpus all on an 8-GPU host with a 4-GPU recipe), NIM runs on the first tp_size GPUs and logs a warning that names the used and idle GPUs; with fewer visible GPUs than tp_size the container does not start.
• GB200: add -e NVIDIA_IMEX_CHANNELS=0 --ulimit memlock=-1; the host needs a configured IMEX domain (one tray, four GPUs).
• Hugging Face instead of a local directory: drop the $MODEL_DIR mount and use -e NIM_MODEL_PATH=hf://deepseek-ai/DeepSeek-V4.1-Flash-1017 -e HF_TOKEN.
• NIM_SERVED_MODEL_NAME is optional: the recipe already sets the served name; passing the same value is harmless.

Step 4 — Verify.

curl --fail http://localhost:8000/v1/health/ready
curl --fail http://localhost:8000/v1/models        # id: deepseek-ai/DeepSeek-V4.1-Flash-1017

Readiness after the weights are cached takes about 10 minutes (CUDA graph capture with DSpark); the first start with hf:// adds the download time.

Override a setting

While NIM_CONFIG_FILE is set, the recipe is authoritative: its keys take precedence over NIM_* environment variables, NIM_PASSTHROUGH_ARGS and command-line options. To run with different settings, start without NIM_CONFIG_FILE and pass the settings explicitly (for example -e NIM_PASSTHROUGH_ARGS="--tp-size 4 --ep-size 4 --context-length 262144 …"). Any change moves the deployment outside the validated configuration.

Format

Each key is an SGLang launch_server flag with - written as _ (true renders the bare flag); the ENV: mapping is exported into the backend process. NIM owns --model-path, --host and --port; the parallel layout (tp_size, pp_size, data_parallel_size, ep_size) and context_length use NIM's profile setting names, so the recipe alone fixes the topology.

License / Terms of Use

GOVERNING TERMS: The recipe files in this resource are NVIDIA-provided configuration for the SGLang Model-Free NIM (Experimental) and are governed by the NVIDIA Software License Agreement and the Product-Specific Terms for NVIDIA AI Products, the same terms as the container they configure.

The model weights are not part of this resource. deepseek-ai/DeepSeek-V4.1-Flash-1017 is a third-party model released by DeepSeek AI under the MIT License; its use is governed by that license in addition to the terms above.

You are responsible for ensuring that your use of any provided models complies with all applicable laws.

Get Help

NVIDIA Developer Community Forum

Get access to community knowledge base articles and support cases (https://forums.developer.nvidia.com/).