## Model Overview

### Description:

Llama 2 SteerLM Chat is a large language model customized using the [SteerLM method](https://arxiv.org/abs/2310.05344)
developed by NVIDIA to allow for user control of model outputs during inference.

### Terms of use

By accessing this model, you are agreeing to the LLama 2 terms and conditions of the [license](https://l.facebook.com/l.php?u=https%3A%2F%2Fai.meta.com%2Fllama%2Flicense&h=AT0DpDKXSE-Yi_pulcu4-5oKNHyiayGrLbQqW7qI_4Ez5y-67Nmyyu38D5NcEmGSifLOtDZAJaQOPMGJg8zCFDJjbYcyJZQPawxWU3-GaVdgStbboYT00yqZvyrxP1DVlbfAog), [acceptable use policy](https://l.facebook.com/l.php?u=https%3A%2F%2Fai.meta.com%2Fllama%2Fuse-policy&h=AT17YXIWa1dxwuzDq96w614_oshaUCWIQYHHHf9BgC0X5UK39DW2tpSP6NzkeadQVs0XBaNlFSo4nydTUcKB5r-Kn4OLs_5IbMEZYAf5hUvyzIwx8-mLom2ic1vw5rSPFUoc9A) and [Meta’s privacy policy](https://www.facebook.com/privacy/policy/)

### References(s):

* [SteerLM method](https://arxiv.org/abs/2310.05344)
* [Llama 2: Open Foundation and Fine-Tuned Chat Models Paper](https://scontent-lhr8-2.xx.fbcdn.net/v/t39.2365-6/10000000_662098952474184_2584067087619170692_n.pdf?_nc_cat=105&ccb=1-7&_nc_sid=3c67a6&_nc_ohc=SvLw337Y0sQAX8zTcqT&_nc_ht=scontent-lhr8-2.xx&oh=00_AfC2GBa-ZPdVLSxOnnjuTZHTDu51NZUcR6KubtLe5tEUbw&oe=64D0333F) 
* [Meta's Llama 2 Webpage](https://ai.meta.com/llama/) 
* [Meta's Llama 2 Model Card](https://github.com/facebookresearch/llama/blob/main/MODEL_CARD.md) 

### Model Architecture:

**Architecture Type:** Transformer 
**Network Architecture:** Llama 2 

### Input:

**Input Format:** Text 
**Input Parameters:** Temperature, Top P = 1, Max Input Tokens: 3072 

### Output:

**Output Format:** Text 
**Output Parameters:** Max Output Tokens: 1024 

### Software Integration:

**Supported Hardware Platform(s):** Ampere/Turing, Hopper 
**Test Hardware** H100, A100 80GB, A100 40GB 
**Supported Operating System(s):** Linux 

### Model Version:

v1.0

## Training & Evaluation:

### Datasets:

**Link:**
* [Open Assistant](https://huggingface.co/datasets/OpenAssistant/oasst1)
* [HelpSteer](https://huggingface.co/datasets/nvidia/HelpSteer)

**Properties (Quantity, Dataset Descriptions, Sensor(s)):** 
* Open Assistant is a human-generated, human-annotated assistant-style conversation corpus consisting of 161,443 messages in 35 different languages, annotated with 461,292 quality ratings, resulting in over 10,000 fully annotated conversation trees.
* HelpSteer is a hybrid human and synthetically-generated, human-annotated dataset of 37,120 prompt-responses built to make more models helpful, factually-correct, coherent, and customizable based on complexity and verbosity.

## Inference:

**Engine:** [Triton](https://developer.nvidia.com/triton-inference-server) 
**Test Hardware:** H100, A100 80GB, A100 40GB 

### Ethical Considerations:

NVIDIA believes Trustworthy AI is a shared responsibility and we have established policies and practices to enable development for a wide array of AI applications. When downloaded or used in accordance with our terms of service, developers should work with their supporting model team to ensure this model meets requirements for the relevant industry and use case and addresses unforeseen product misuse. For more detailed information on ethical considerations for this model, please see the Model Card++ Explainability, Bias, Safety & Security, and Privacy Subcards. Please report security vulnerabilities or NVIDIA AI Concerns [here](https://www.nvidia.com/en-us/support/submit-security-vulnerability/).

llama2-70b-steerlm

Llama 2 SteerLM Chat is a large language model, aligned using the SteerLM technique developed by NVIDIA. This allows you to adjust the preferred style of response to attributes (such as creativity, complexity and verbosity) at inference time.

NV-Llama2-70B-SteerLM-Chat