Model Overview
--------------

### Description:
Gemma-7B is a 7B parameter model from Gemma family of models from Google. It has been instruction-tuned so it can respond to prompts in a conversation manner.
Nvidia has converted original Gemma weights and format into weight and format that can be consumed by Tensorrt-LLM.


### Terms of use:
By accessing this model, you are agreeing to [Gemma Terms of Use](https://ai.google.dev/gemma/terms), [Gemma Prohibited Use Policy](https://ai.google.dev/gemma/prohibited_use_policy) .

### References(s):

* Gemma [Model Card](https://ai.google.dev/gemma/docs/model_card)  
* Gemma [blogpost](https://blog.google/technology/developers/gemma-open-models/)


### Input:
**Input Format:** Text  

**Input Parameters:** None 

### Output:

**Output Format:** Text

**Output Parameters:** None

### Software Integration:

**Supported Hardware Platform(s):** RTX 4090  

**Supported Operating System(s):** Windows

### Inference:

[Windows Setup with TRT-LLM](https://github.com/NVIDIA/TensorRT-LLM/tree/gemma/windows).

[TRT-LLM Inference Engine](https://github.com/NVIDIA/TensorRT-LLM/tree/gemma/examples/gemma)  


### Test Hardware:
RTX 4090

gemma-7b-fp16-rtx

Gemma-7B is a 7B parameter model from Gemma family of models from Google. It has been instruction-tuned so it can respond to prompts in a conversation manner.

Gemma-7B-FP16-RTX

Model Overview

Description:

Terms of use:

References(s):

Input:

Output:

Software Integration:

Inference:

Test Hardware: