NGC Catalog
CLASSIC
Welcome Guest
Containers
Llama-3.1-70b-instruct PB October 2024 (PB 24h2)

Llama-3.1-70b-instruct PB October 2024 (PB 24h2)

For copy image paths and more information, please view on a desktop device.
Logo for Llama-3.1-70b-instruct PB October 2024 (PB 24h2)
Associated Products
Features
Description
Llama 3.1 70B-Instruct NIM Production Branch October 2024 (PB 24h2) offers a 9-month lifecycle for API stability, with monthly patches for high and critical software vulnerabilities.
Publisher
NVIDIA
Latest Tag
1.3.6
Modified
May 4, 2025
Compressed Size
7.3 GB
Multinode Support
No
Multi-Arch Support
Yes
1.3.6 (Latest) Security Scan Results

Linux / amd64

Sorry, your browser does not support inline SVG.

What Is NVIDIA NIM?

NVIDIA NIM, part of NVIDIA AI Enterprise, is a set of easy-to-use microservices designed to speed up generative AI deployment in enterprises. Supporting a wide range of AI models, including NVIDIA AI foundation and custom models, it ensures seamless, scalable AI inferencing, on-premises or in the cloud, leveraging industry standard APIs.

The Llama 3.1 70B-Instruct NIM simplifies the deployment of the Llama 3.1 70B instruction tuned model which is optimized for language understanding, reasoning, and text generation use cases, and outperforms many of the available open source chat models on common industry benchmarks.

NVIDIA NIM offers prebuilt containers for large language models (LLMs) that can be used to develop chatbots, content analyzers—or any application that needs to understand and generate human language. Each NIM consists of a container and a model and uses a CUDA-accelerated runtime for all NVIDIA GPUs, with special optimizations available for many configurations. Whether on-premises or in the cloud, NIM is the fastest way to achieve accelerated generative AI inference at scale.

What Is Llama 3.1 70B-Instruct NIM Production Branch October 2024?

The Llama 3.1 70B-Instruct NIM Production Branch, exclusively available with NVIDIA AI Enterprise, is a 9-month supported, API-stable branch that includes monthly fixes for high and critical software vulnerabilities. This branch provides a stable and secure environment for building your mission-critical AI applications. The Llama 3.1 70B-Instruct NIM production branch releases every six months with a three-month overlap in between two releases.

Getting started with Llama 3.1 70B-Instruct NIM Production Branch October 2024

Before you start, ensure that your environment is set up by following one of the deployment guides available in the NVIDIA AI Enterprise Documentation.

Deploying and integrating NVIDIA NIM is straightforward thanks to our industry standard APIs. Visit the NIM Container LLM page for release documentation, deployment guides and more.

Security Vulnerabilities in Open Source Packages

Please review the Security Scanning (LINK) tab to view the latest security scan results.

For certain open-source vulnerabilities listed in the scan results, NVIDIA provides a response in the form of a Vulnerability Exploitability eXchange (VEX) document. The VEX information can be reviewed and downloaded from the Security Scanning (LINK) tab.

Get Help

Enterprise Support

Get access to knowledge base articles and support cases or submit a ticket.

NVIDIA AI Enterprise Documentation

Visit the NVIDIA AI Enterprise Documentation Hub for release documentation, deployment guides and more.

Governing Terms

The NIM container is governed by the NVIDIA Software License Agreement; and the Product Specific Terms for AI Products; and the use of this model is governed by the NVIDIA AI Foundation Models Community License Agreement. ADDITIONAL INFORMATION: Llama 3.1 Community License Agreement, Built with Llama.

You are responsible for ensuring that your use of NVIDIA AI Foundation Models complies with all applicable laws.