## Model Overview

### Description:

The Mamba-Chat model is the first chat language model based on a state-space model architecture, not a transformer.

### Terms of use:

By using this model, you are agreeing to the terms and conditions of the [license](https://github.com/havenhq/mamba-chat?tab=Apache-2.0-1-ov-file)

### References(s):

[Mamba paper](https://arxiv.org/abs/2312.00752) 
[Mamba-Chat GitHub repository](https://github.com/havenhq/mamba-chat) 
[Mamba-Chat on HuggingFace](https://huggingface.co/havenhq/mamba-chat) 

### Model Architecture:

**Architecture Type** State-space model 
**Network Architecture:** Mamba 

### Input:

**Input Format:** Text 
**Input Parameters:** temperature, top_k, top_p, max_tokens, (optional) stop, bad, seed 
**Other Properties Related to Input:** None 

### Output:

**Output Format:** Text 
**Output Parameters:** None 
**Other Properties Related to Output:** stream 

### Supported Operating System(s):

Linux

## Inference:

**Engine:** Triton 
**Test Hardware:** L40

mamba-chat

Mamba-Chat is a state-of-the-art AI model designed for efficient sequence modeling. The model can be used for text generation and chat applications

The list of completion choices the model generated for the input prompt.

Choices

Usage statistics for the completion request.

ChatCompletion

ChatCompletionChunk

A word or list of words not to use. The words are case sensitive.

The maximum number of tokens to generate in any given call. Note that the model is not aware of this value, and generation will simply stop at the number of tokens specified.

Max Tokens

A list of messages comprising the conversation so far.

Messages

If specified, our system will make a best effort to sample deterministically, such that repeated requests with the same seed and parameters should return the same result.

Seed

A string or a list of strings where the API will stop generating further tokens. The returned text will not contain the stop sequence.

Stop

If set, partial message deltas will be sent. Tokens will be sent as data-only server-sent events (SSE) as they become available (JSON responses are prefixed by `data: `), with the stream terminated by a `data: [DONE]` message.

Stream

The sampling temperature to use for text generation. The higher the temperature value is, the less deterministic the output text will be. It is not recommended to modify both temperature and top_p in the same call.

Temperature

The top-k most likely tokens to sample from. If top_k = 0, don't limit the number of candidate tokens. top_k and top_p can be used together. If top_k > 0 and top_p > 0, then top_k is applied first, then top_p.

Top K

The top-p sampling mass used for text generation. The top-p value determines the probability mass that is sampled at sampling time. For example, if top_p = 0.2, only the most likely tokens (summing to 0.2 cumulative probability) will be sampled. It is not recommended to modify both temperature and top_p in the same call.

Top P

ChatRequest

The reason the model stopped generating tokens. This will be `stop` if the model hit a natural stop point or a provided stop sequence, or `length` if the maximum number of tokens specified in the request was reached.

Finish Reason

The index of the choice in the list of choices (always 0).

Index

A chat completion message generated by the model.

Choice

A chat completion delta generated by streamed model responses.

ChoiceChunk

Function instance used to invoke the request

InvokeError

Content

Role

Message

Number of tokens in the generated completion.

Completion Tokens

Prompt Tokens

Total number of tokens used in the request (prompt + completion).

Total Tokens

Usage

NVCF function

Result is pending. Client should poll using the requestId.


Result is in a different region. Client should use the fully-qualified
endpoint specified in 'Location' response header to fetch the result.
Client can use the same Bearer token in 'Authorization' header when
retrieving the result from the redirected region.


Validation failed, provided entity could not be processed.

requestId

Gets the result of an earlier function invocation request that returned a status of 202.