## Model Overview

### Description:

Llama 2 is a large language AI model comprising a collection of models capable of generating text and code in response to prompts.

### Terms of use

By accessing this model, you are agreeing to the Llama 2 terms and conditions of the [license](https://l.facebook.com/l.php?u=https%3A%2F%2Fai.meta.com%2Fllama%2Flicense&h=AT0DpDKXSE-Yi_pulcu4-5oKNHyiayGrLbQqW7qI_4Ez5y-67Nmyyu38D5NcEmGSifLOtDZAJaQOPMGJg8zCFDJjbYcyJZQPawxWU3-GaVdgStbboYT00yqZvyrxP1DVlbfAog), [acceptable use policy](https://l.facebook.com/l.php?u=https%3A%2F%2Fai.meta.com%2Fllama%2Fuse-policy&h=AT17YXIWa1dxwuzDq96w614_oshaUCWIQYHHHf9BgC0X5UK39DW2tpSP6NzkeadQVs0XBaNlFSo4nydTUcKB5r-Kn4OLs_5IbMEZYAf5hUvyzIwx8-mLom2ic1vw5rSPFUoc9A) and [Meta’s privacy policy](https://www.facebook.com/privacy/policy/)

### References(s):

* [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://scontent-lhr8-2.xx.fbcdn.net/v/t39.2365-6/10000000_662098952474184_2584067087619170692_n.pdf?_nc_cat=105&ccb=1-7&_nc_sid=3c67a6&_nc_ohc=SvLw337Y0sQAX8zTcqT&_nc_ht=scontent-lhr8-2.xx&oh=00_AfC2GBa-ZPdVLSxOnnjuTZHTDu51NZUcR6KubtLe5tEUbw&oe=64D0333F) paper 
* [Meta's Llama 2](https://ai.meta.com/llama/) webpage 
* [Meta's Llama 2 Model Card](https://github.com/facebookresearch/llama/blob/main/MODEL_CARD.md) webpage 

### Model Architecture:

**Architecture Type:** Transformer 
**Network Architecture:** Llama 2 
**Model version:** N/A 

### Input:

**Input Format:** Text 
**Input Parameters:** Temperature, TopP 

### Output:

**Output Format:** Text 
**Output Parameters:** Max output tokens 

### Software Integration:

**Runtime(s):** N/A 
**Supported Hardware Platform(s):** Hopper, Ampere/Turing 
**Supported Operating System(s):** Linux 

## Training & Finetuning:

### Dataset:

**Link:**
* [Falcon RefinedWeb](https://arxiv.org/abs/2306.01116)
* [BigCode - The Stack (StarCoder)](https://huggingface.co/datasets/bigcode/the-stack)
* [Wikipedia](https://www.tensorflow.org/datasets/catalog/wikipedia)
* [ArXiv](https://arxiv.org/pdf/2101.00027.pdf)
* [Books3](https://arxiv.org/pdf/2101.00027.pdf)
* [StackExchange](https://archive.org/details/stackexchange) 

**Properties (Quantity, Dataset Descriptions, Sensor(s)):** 
* Falcon RefinedWeb is a massive English web dataset containing 500-650GT depending on the used tokenizer
* The Stack dataset is a collection of source code in over 300 programming languages
* Wikipedia is 20.54GB of online encyclopedia written and maintained by a community of volunteers
* The Pile: ArXiv is 56.21GB of ArXiv papers contents
* The Pile: Books3 is 100.96GB dataset of books derived from a copy of the contents of the Bibliotik private tracker made available by Shawn Presser
* StackExchange is an anonymized dump of all user-contributed content on the Stack Exchange network where each site is formatted as a separate archive consisting of zipped XML files

## Inference:

**Engine:** [Triton](https://developer.nvidia.com/triton-inference-server) 
**Test Hardware:** Other

llama2-13b

Llama 2 is a large language AI model capable of generating text and code in response to prompts.

Llama 2 13B