Model Overview

Terms of use

GOVERNING TERMS: By accessing this engine, you are agreeing to NVIDIA TensorRT Engine License.

ATTRIBUTION: Meta license and Meta Acceptable Use Policy available at https://llama.meta.com/llama-downloads/.

Third-Party Community Consideration

This model is not owned or developed by NVIDIA. This model has been developed and built to a third-party’s requirements for this application and use case.

Description:

Llama 2 is a large language AI model comprising a collection of models capable of generating text and code in response to prompts. TensorRT engines optimizing Llama 2 are generated by TensorRT-Cloud.

These TensorRT engines are ready for commercial use.

References(s):

• Llama 2: Open Foundation and Fine-Tuned Chat Models paper
  
• Meta's Llama 2 webpage
  
• Meta's Llama 2 Model Card webpage

Model Architecture:

Architecture Type: Transformer
Network Architecture: Llama 2
Model version: N/A

Input:

Input Format: Text
Input Parameters: Temperature, TopP
Other Properties Related to Output: None

Output:

Output Format: Text
Output Parameters: Max output tokens
Other Properties Related to Output: None

Software Integration:

Check the version tag for engine support matrix

Training & Finetuning:

Dataset:

Llama 2 was pretrained on 2 trillion tokens of data from publicly available sources. The fine-tuning data includes publicly available instruction datasets, as well as over one million new human-annotated examples. Training Data

Inference:

TRT-LLM Inference Engine

Ethical Considerations:

NVIDIA believes Trustworthy AI is a shared responsibility and we have established policies and practices to enable development for a wide array of AI applications. When downloaded or used in accordance with our terms of service, developers should work with their internal model team to ensure this model meets requirements for the relevant industry and use case and addresses unforeseen product misuse. Please report security vulnerabilities or NVIDIA AI Concerns here.