ContainersNVIDIADeepSeek-R1NVIDIALlama-3.1-Nemotron-70B-InstructNVIDIAPyTorch
CollectionsNVIDIAOmniverse Kit (FB)NVIDIADeepStream SDKNVIDIAOmniverse Kit App Streaming
ModelsNVIDIAStyleGAN3 pretrained modelsNVIDIAPeopleNetNVIDIATrafficCamNet
ResourcesNVIDIARiva Skills Quick Start
Helm ChartsNVIDIAGPU OperatorNVIDIANVIDIA NIM Operator
NVIDIA
NVIDIA
Llama-3.1-8B-Instruct
Container
NVIDIA
NVIDIA
Llama-3.1-8B-Instruct

NVIDIA NIM for GPU accelerated Llama-3.1-8B-Instruct inference through OpenAI compatible APIs

NVIDIA AI Enterprise SupportedNVIDIA NIM
LayerLabelCreated
sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4CMD
/opt/nim/start_server.sh
06/02/2026 5:05 PM UTC
sha256:0cd50139531bb8e2bab28b8fc0dbeabe4ccb3f3d1525cffa4e5e1d342a132e8eCOPY
--chown=nim:nim NOTICE_2 /opt/nim/NOTICE_2
06/02/2026 5:05 PM UTC
sha256:715d0f264e98c5b25e969684a5fbfda75c7851c926c42f478eb4c872edcd827aCOPY
--chown=nim:nim NOTICE /opt/nim/NOTICE
06/02/2026 5:05 PM UTC
sha256:9aad6e1f82f1a599a37d834ed94b86dabc35d2673ee9b9629ca33457538c7cf5COPY
--chown=nim:nim MODEL_LICENSE /opt/nim/MODEL_LICENSE
06/02/2026 5:05 PM UTC
sha256:a2f34dc3f7c83b7dd2878a49cb373edf4cb7dfd4dc910b956e628b78e0df80abCOPY
--chown=nim:nim model_manifest.yaml /opt/nim/etc/default/model_manifest.yaml
06/02/2026 5:05 PM UTC
sha256:854e98ae6231754cb6b90296c6a0246e747f7f4cc1aa803a997a1b9e96ec0795COPY
--chown=nim:nim VERSION /opt/nim/VERSION
06/02/2026 5:05 PM UTC
sha256:0b7fe40fa3226e2bf8077cceef9d5d1d4edd7b54fcc2bf0bdab29fae7d0ec6b6COPY
--chown=nim:nim LICENSE /opt/nim/LICENSE
06/02/2026 5:05 PM UTC
sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4ENV
INFERENCE_MICROSERVICE_LLM_NIM_VERSION=2.0.6
06/02/2026 5:05 PM UTC
sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4ENV
NIM_SERVED_MODEL_NAME=meta/llama-3.1-8b-instruct
06/02/2026 5:05 PM UTC
sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4ENV
NIM_HTTP_API_PORT=8000
06/02/2026 5:05 PM UTC
...