Beam 1.0 NIM Serving Recipes

NIM tuning configs (NIM_CONFIG_FILE) for Beam 1.0 (501B total / 23B active MoE), low-latency strategy: tensor parallelism with the DFlash2 speculative drafter (block size 8).

Image: nvcr.io/nim/nvidia/sglang-model-free-nim-experimental:2.1.7-variant-beam-1.0-501b (amd64 and arm64)

Format

Each key is an sglang serve flag with - written as _ (true renders the bare flag); the optional ENV: mapping is exported into the backend environment. NIM owns --model-path, --served-model-name, --host and --port, so those come from NIM_MODEL_PATH / NIM_SERVED_MODEL_NAME instead. The parallel layout (tp_size, pp_size, data_parallel_size, ep_size) and context_length use the setting names of NIM's profiles, so the recipe alone fixes the topology. The DFlash2 drafter is referenced by its Hugging Face id and is downloaded into the NIM cache at startup (about 3.6 GB).

Launch

Download the recipes with the NGC CLI (ngc config set with an NGC API key that can read this resource) and pick the file for your GPUs from the table below. NGC CLI saves files as 0600, so make the recipe readable by the container user (nvs, UID 1000):

ngc registry resource download-version "nim/nvidia/beam-1.0-recipe:2.1.7-beam-1.0-501b"
RECIPE=$PWD/beam-1.0-recipe_v2.1.7-beam-1.0-501b/beam-1.0_h200_latency_fp8_tp8.yaml
chmod 0644 "$RECIPE"

The recipe fixes the whole topology; the launch only names the model and the recipe (NIM_SERVED_MODEL_NAME is optional and keeps the served name beam). NIM_MODEL_PATH=hf://... downloads the weights from Hugging Face into the NIM cache on first start (pass HF_TOKEN to avoid anonymous rate limits):

IMAGE=nvcr.io/nim/nvidia/sglang-model-free-nim-experimental:2.1.7-variant-beam-1.0-501b
docker run -d --name beam-nim --gpus all --ipc=host \
  --ulimit memlock=-1 --ulimit stack=67108864 -p 8000:8000 \
  -v beam-nim-cache:/opt/nim/.cache \
  -v "$RECIPE":/opt/nim/recipe.yaml:ro \
  -e HF_TOKEN \
  -e NIM_MODEL_PATH=hf://reflection/Beam-1.0-501B-A23B-FP8 \
  -e NIM_CONFIG_FILE=/opt/nim/recipe.yaml \
  -e NIM_SERVED_MODEL_NAME=beam \
  "$IMAGE" nim-serve

On an 8-GPU B200 host, pick four GPUs with --gpus '"device=4,5,6,7"'.

No NIM_* override or NIM_MODEL_PROFILE is needed: NIM selects the profile the recipe describes. list-model-profiles with the same mounts shows the profile and its GPU memory figure.

To use weights already on disk instead of downloading from Hugging Face, mount the directory (for example -v /path/to/Beam-1.0-501B-A23B-FP8:/models/model:ro), set NIM_MODEL_PATH=/models/model, and make it readable with chmod -R a+rX.

H100

TP8 FP8 puts about 60 GB of target weights and 1.4 GB of DFlash2 drafter on each 80 GB GPU, so the H100 file sets mem_fraction_static to 0.88 and context_length to 262144. A single prompt close to 256K tokens can exceed the KV cache capacity; the full 1M context needs GPUs with more memory (H200).

GB200

The GB200 file sets the NCCL multi-node NVLink options and limits CUDA graphs to decode batch sizes up to 64 (no prefill graphs). Launch it with the IMEX channel and the two ulimits:

docker run ... --ulimit memlock=-1 --ulimit stack=67108864 \
  -e NVIDIA_IMEX_CHANNELS=0 ... "$IMAGE" nim-serve

On some GB200 hosts the driver refuses to instantiate CUDA graphs inside a container (the log shows Failed to create CPU mapping, or the first request fails with CUDA-capable device(s) is/are busy or unavailable). Add --privileged to the docker run command on such hosts.

SGLang's symmetric memory and NVLink multicast need IMEX on GB200, even inside one tray: run nvidia-imex on the host (a single-node domain is enough for TP4) and pass NVIDIA_IMEX_CHANNELS=0 so the container gets /dev/nvidia-caps-imex-channels/channel0.

Query: curl localhost:8000/v1/chat/completions -H 'Content-Type: application/json' -d '{"model":"beam","messages":[{"role":"user","content":"Hello"}]}'

License/Terms of Use

GOVERNING TERMS: The NIM container is governed by the NVIDIA Software License Agreement and the Product-Specific Terms for NVIDIA AI Products.

The use of any models with this container is governed by the NVIDIA Open Model Agreement in addition to any terms which govern the specific models used.

These recipes are validated for Beam 1.0 (reflection/Beam-1.0-501B-A23B-FP8, reflection/Beam-1.0-501B-A23B-NVFP4 and the reflection/Beam-1.0-501B-A23B-DFlash2 drafter). NVIDIA does not provide these models; their use is governed by the Apache License 2.0.

You are responsible for ensuring that your use of any provided models complies with all applicable laws.

Get Help

Getting started with the NIM

Deploying and integrating the NIM is straightforward thanks to our industry standard APIs. Visit the NIM Container page for release documentation, deployment guides and more NIM Container LLM.

Enterprise Support

Get access to knowledge base articles and support cases or submit a ticket.