NGC | Catalog
CatalogModelsMistral-7B Chat Int4

Mistral-7B Chat Int4

Logo for Mistral-7B Chat Int4
Description
The Mistral-7B-Instruct-v0.1 Large Language Model (LLM) is a instruct fine-tuned version of the Mistral-7B-v0.1 generative text model using a variety of publicly available conversation datasets.
Publisher
Mistral.ai
Latest Version
1.1
Modified
February 13, 2024
Size
12.5 GB

Model Overview

Description:

The Mistral-7B-Instruct-v0.1 Large Language Model (LLM) is a instruct fine-tuned version of the Mistral-7B-v0.1 generative text model using a variety of publicly available conversation datasets. Mistral-7B is released under the Apache 2.0 license

This instruction model is a transformer model with the following architecture choices:

  • Grouped-Query Attention
  • Sliding-Window Attention
  • Byte-fallback BPE tokenizer

Terms of use:

By accessing this model, you are agreeing to the Mistral 7B Terms and Conditions of the License, Terms of Service.

References(s):

Model Architecture:

Architecture Type: Transformer

Network Architecture: Mistral-7B

Input:

Input Format: Text

Input Parameters: None

Output:

Output Format: Text

Output Parameters: None

Software Integration:

Supported Hardware Platform(s): RTX 4090, Ada GPUs

Supported Operating System(s): Windows

Inference:

TRT-LLM Inference Engine
Windows Setup with TRT-LLM

Test Hardware:

RTX 4090