Skip to main content
NVIDIA
SGLang Model-Free NIM Experimental
Container
NVIDIA
SGLang Model-Free NIM Experimental

SGLang Model-Free NIM is a production-ready inference microservice that serves a broad range of LLMs supplied at runtime in Hugging Face format, with automatic backend selection and optimized configurations.

LayerLabelCreated
sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4ENTRYPOINT
/opt/nim/start_server.sh
10/20/2026 8:04 AM UTC
sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4USER
nvs:1000
10/20/2026 8:04 AM UTC
sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1WORKDIR
/opt/nim
10/20/2026 8:04 AM UTC
sha256:618b838bcb0055d731afd4b5f18096a27039a6b39c768d59214d9dc2c4fa162aRUN
ENGINE_SOURCE_COMMIT=deca58f7ac0d0ab527aa1a765643be5845fc1a7c SGLANG_VERSION_PUBLIC=0.5.22.dev0 INSTALL_DEBUG_TOOLS=0 /bin/bash -lc set -euo pipefail; rm -rf /opt/nim/runtime-cache/sglang/.cache /root/.cache/flashinfer /root/.cache/sglang /root/.cache/pip /root/.triton; find /sgl-workspace/sglang/python /usr/local/lib/python3.12/dist-packages /opt/nim -type f \( -name '*.orig' -o -name '*.rej' \) -print -delete | sed 's#^#strip patch backup: #'; test -z "$(find /sgl-workspace/sglang/python /usr/local/lib/python3.12/dist-packages /opt/nim       -type f \( -name '*.orig' -o -name '*.rej' \) -print -quit)"; rm -rf /opt/nim/patches /opt/nim/vendor; test ! -e /opt/nim/patches; test ! -e /opt/nim/vendor; chown -R 1000:1000 /opt/nim/runtime-cache; test -z "$(find /opt/nim/runtime-cache -not -user 1000 -print -quit)"; test -x /opt/nim/start_server.sh
10/20/2026 8:04 AM UTC
sha256:f2db83790385eaf6ca56cd793873d7d129cd597fb0a7dd95e94ef12265e1ff7aRUN
ENGINE_SOURCE_COMMIT=deca58f7ac0d0ab527aa1a765643be5845fc1a7c SGLANG_VERSION_PUBLIC=0.5.22.dev0 INSTALL_DEBUG_TOOLS=0 /bin/bash -lc set -euo pipefail; bash /opt/nim/patches/gate/image_canaries.sh; HF_HOME=/tmp/gate-hf-home python3 /opt/nim/patches/gate/import_gate.py; rm -rf /tmp/gate-hf-home
10/20/2026 8:04 AM UTC
sha256:39d6c76e6ca9049231456be39597f542c466cbe0f25ec950f11f10b842c7cdf8COPY
VERSION /opt/nim/VERSION
10/20/2026 8:04 AM UTC
sha256:ffb1a609e18c8d0de5d7ad857d2c626dc61b6784f362068ef76e50bcfee4782fRUN
ENGINE_SOURCE_COMMIT=deca58f7ac0d0ab527aa1a765643be5845fc1a7c SGLANG_VERSION_PUBLIC=0.5.22.dev0 INSTALL_DEBUG_TOOLS=0 /bin/bash -lc set -euo pipefail; patch_root=/opt/nim/patches/055; cd /sgl-workspace/sglang; patch --batch --forward --fuzz=0 --dry-run -p1 < "${patch_root}/055-vl-processor-ignores-checkpoint-code.patch"; patch --batch --forward --fuzz=0 -p1 < "${patch_root}/055-vl-processor-ignores-checkpoint-code.patch"; python3 -m compileall -q python/sglang/srt/utils/hf_transformers/processor.py test/registered/unit/utils/test_hf_transformers_loading.py; grep -q 'def _get_builtin_vl_processor' python/sglang/srt/utils/hf_transformers/processor.py; HF_HOME=/tmp/probe-055-hf-home python3 "${patch_root}/probe.py"; rm -rf /tmp/probe-055-hf-home
10/20/2026 8:04 AM UTC
sha256:8a4514f546f42ccef5e6fd71c3a55cfc5969567ce753d663f1941ad3c5f54812RUN
ENGINE_SOURCE_COMMIT=deca58f7ac0d0ab527aa1a765643be5845fc1a7c SGLANG_VERSION_PUBLIC=0.5.22.dev0 INSTALL_DEBUG_TOOLS=0 /bin/bash -lc set -euo pipefail; patch_root=/opt/nim/patches/054; py_site="$(python3 -c 'import site; print(site.getsitepackages()[0])')"; cd "${py_site}"; patch --batch --forward --fuzz=0 --dry-run -p1 < "${patch_root}/patch.diff"; patch --batch --forward --fuzz=0 -p1 < "${patch_root}/patch.diff"; python3 -m compileall -q nim_stack/cli/serve.py nim_stack/engines/base.py nim_stack/engines/sglang/adapter.py nim_stack/engines/sglang/context_limit.py nim_stack/manifest/model_metadata.py; grep -q 'def fit_context_length_to_model' nim_stack/engines/sglang/context_limit.py; grep -q 'self._adapter.fit_to_model_files(' nim_stack/cli/serve.py; grep -q 'def _fetch_metadata_files_directly' nim_stack/manifest/model_metadata.py; HF_HOME=/tmp/probe-054-hf-home python3 "${patch_root}/probe.py"; rm -rf /tmp/probe-054-hf-home
10/20/2026 8:04 AM UTC
sha256:4556726d93c53794a84639c5ed8719802ed288990ea915c828c0941a88f20073RUN
ENGINE_SOURCE_COMMIT=deca58f7ac0d0ab527aa1a765643be5845fc1a7c SGLANG_VERSION_PUBLIC=0.5.22.dev0 INSTALL_DEBUG_TOOLS=0 /bin/bash -lc set -euo pipefail; patch_root=/opt/nim/patches/053; cd /sgl-workspace/sglang; patch --batch --forward --fuzz=0 --dry-run -p1 < "${patch_root}/053-constrained-decoding-forces-thinking.patch"; patch --batch --forward --fuzz=0 -p1 < "${patch_root}/053-constrained-decoding-forces-thinking.patch"; python3 -m compileall -q python/sglang/srt/entrypoints/openai/serving_chat.py test/registered/unit/entrypoints/openai/test_serving_chat.py; grep -q 'def _uses_constrained_decoding' python/sglang/srt/entrypoints/openai/serving_chat.py; grep -q 'Constrained decoding opens the reasoning first' python/sglang/srt/entrypoints/openai/serving_chat.py; python3 "${patch_root}/probe.py"
10/20/2026 8:03 AM UTC
sha256:2dbf6fd5902a027209a94977864eccd07539a0c3ec14a1330b36a1f04fc564e4RUN
ENGINE_SOURCE_COMMIT=deca58f7ac0d0ab527aa1a765643be5845fc1a7c SGLANG_VERSION_PUBLIC=0.5.22.dev0 INSTALL_DEBUG_TOOLS=0 /bin/bash -lc set -euo pipefail; patch_root=/opt/nim/patches/052; py_site="$(python3 -c 'import site; print(site.getsitepackages()[0])')"; cd "${py_site}"; patch --batch --forward --fuzz=0 --dry-run -p1 < "${patch_root}/patch.diff"; patch --batch --forward --fuzz=0 -p1 < "${patch_root}/patch.diff"; python3 -m compileall -q nim_stack/engines/sglang/lifecycle.py; grep -q 'def _reattach_stdin' nim_stack/engines/sglang/lifecycle.py; grep -q '^    _reattach_stdin()$' nim_stack/engines/sglang/lifecycle.py; python3 "${patch_root}/probe.py"
10/20/2026 8:03 AM UTC
1-10 of 196 items

Modal Content

NVIDIA uses cookies to improve your experience on our web site. We and our third-party partners also use cookies and other tools to collect and record information you provide as well as information about your interactions with our websites for performance improvement, analytics, and to assist in marketing efforts. By clicking "Accept All", you consent to our use of cookies and other tools as described in our Cookie Policy. You can manage your cookie settings by clicking on "Manage Settings." By continuing to use this site or by clicking one of the buttons below, you agree to our Terms of Service (which contains important waivers). Please see our Privacy Policy for more information on our privacy practices.