Llama-3.2-3B-Instruct

NVIDIA

Container

NVIDIA

Llama-3.2-3B-Instruct

NVIDIA NIM for GPU accelerated Llama-3.2-3B-Instruct inference through OpenAI compatible APIs

NVIDIA NIM

NIM MetadataTo verify your workspace, run the Verify CLI locally and compare the generated hash with the NGC-published hash shown here. Learn more in the NVIDIA Documentation.

			Profile	Engine		Actions
H100 80GB HBM3	2330:10de	fp8	throughput-lora	tensorrt_llm	1	Copy NIM CLI Verification
L40S	26b9:10de	fp8	throughput-lora	tensorrt_llm	1	Copy NIM CLI Verification
H200	2335:10de	fp8	throughput-lora	tensorrt_llm	1	Copy NIM CLI Verification
A10G	2237:10de	fp16	throughput-lora	tensorrt_llm	1	Copy NIM CLI Verification
H100 NVL	2321:10de	fp8	throughput-lora	tensorrt_llm	1	Copy NIM CLI Verification
H200	2335:10de	fp16	throughput-lora	tensorrt_llm	1	Copy NIM CLI Verification
A100 SXM4 80GB	20b2:10de	fp16	throughput	tensorrt_llm	1	Copy NIM CLI Verification
H100 NVL	2321:10de	fp16	throughput	tensorrt_llm	1	Copy NIM CLI Verification
H100 NVL	2321:10de	fp16	throughput-lora	tensorrt_llm	1	Copy NIM CLI Verification
H100 80GB HBM3	2330:10de	fp16	throughput-lora	tensorrt_llm	1	Copy NIM CLI Verification