Model Overview
--------------

### Description:

Llama 2 is a large language AI model comprising a collection of models capable of generating text and code in response to prompts.

### Terms of use

By accessing this model, you are agreeing to the LLama 2 terms and conditions of the [license](https://l.facebook.com/l.php?u=https%3A%2F%2Fai.meta.com%2Fllama%2Flicense&h=AT0DpDKXSE-Yi_pulcu4-5oKNHyiayGrLbQqW7qI_4Ez5y-67Nmyyu38D5NcEmGSifLOtDZAJaQOPMGJg8zCFDJjbYcyJZQPawxWU3-GaVdgStbboYT00yqZvyrxP1DVlbfAog), [acceptable use policy](https://l.facebook.com/l.php?u=https%3A%2F%2Fai.meta.com%2Fllama%2Fuse-policy&h=AT17YXIWa1dxwuzDq96w614_oshaUCWIQYHHHf9BgC0X5UK39DW2tpSP6NzkeadQVs0XBaNlFSo4nydTUcKB5r-Kn4OLs_5IbMEZYAf5hUvyzIwx8-mLom2ic1vw5rSPFUoc9A) and [Meta’s privacy policy](https://www.facebook.com/privacy/policy/)

### References(s):

*   [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://ai.meta.com/research/publications/llama-2-open-foundation-and-fine-tuned-chat-models/) paper  
    
*   [Meta's Llama 2](https://ai.meta.com/llama/) webpage  
    
*   [Meta's Llama 2 Model Card](https://github.com/facebookresearch/llama/blob/main/MODEL_CARD.md) webpage  
    

### Model Architecture:

**Architecture Type:** Transformer  
**Network Architecture:** Llama 2  
**Model version:** N/A  

### Input:

**Input Format:** Text  
**Input Parameters:** Temperature, TopP  
**Other Properties Related to Output:** None  

### Output:

**Output Format:** Text  
**Output Parameters:** Max output tokens  
**Other Properties Related to Output:** None  

### Software Integration:

**Runtime(s):** N/A  
**Supported Hardware Platform(s):** RTX 4090  
**Supported Operating System(s):** Windows

Training & Finetuning:
----------------------

### Dataset:
Llama 2 was pretrained on 2 trillion tokens of data from publicly available sources. The fine-tuning data includes publicly available instruction datasets, as well as over one million new human-annotated examples. [Training Data](https://github.com/facebookresearch/llama/blob/main/MODEL_CARD.md#training-data)

### Inference:

[TRT-LLM Inference Engine](https://github.com/NVIDIA/TensorRT-LLM/tree/main/examples/llama)  
[Windows Setup with TRT-LLM](https://github.com/NVIDIA/TensorRT-LLM/tree/main/windows) 

### Test Hardware:
RTX 4090

llama2-13b

LlaMa 2 is a large language AI model capable of generating text and code in response to prompts.

Llama2-13b Chat Int4