Llama-3.1-8B-Instruct

NVIDIA

Container

NVIDIA

Llama-3.1-8B-Instruct

NVIDIA NIM for GPU accelerated Llama-3.1-8B-Instruct inference through OpenAI compatible APIs

NIM MetadataTo verify your workspace, run the Verify CLI locally and compare the generated hash with the NGC-published hash shown here. Learn more in the NVIDIA Documentation.Get detailed NIM model metadata, including properties, deployment, and configuration, via the NIM Metadata API.

			Engine		Actions
a100_sxm4_40gb	20b0:10de	bf16	tensorrt_llm	2	Copy NIM CLI Verification
a100	20b2:10de	bf16	tensorrt_llm	2	Copy NIM CLI Verification
a100	20b2:10de	bf16	tensorrt_llm	1	Copy NIM CLI Verification
gb200	2941:10de	fp8	tensorrt_llm	2	Copy NIM CLI Verification
gh200_144gb	2348:10de	fp8	tensorrt_llm	1	Copy NIM CLI Verification
h100_nvl	2321:10de	bf16	tensorrt_llm	1	Copy NIM CLI Verification
h100	2330:10de	bf16	tensorrt_llm	1	Copy NIM CLI Verification
h200_nvl	233b:10de	bf16	tensorrt_llm	1	Copy NIM CLI Verification
—	—	bf16	vllm	4	Copy NIM CLI Verification
h100	2330:10de	fp8	tensorrt_llm	2	Copy NIM CLI Verification