## Model Overview

### Description:

Phi-2 is a 2.7 billion parameter language model developed by Microsoft Research. The phi-2 model is best suited for prompts using the QA format, the chat format, and the code format.

### Third-Party Community Consideration

This model is not owned or developed by NVIDIA. This model has been developed and built to a third-party’s requirements for this application and use case; see link to Non-NVIDIA Model Card.
By using the Phi-2 model, you are agreeing to the terms and conditions of the [license](https://huggingface.co/microsoft/phi-2/resolve/main/LICENSE), acceptable use policy and Microsoft's privacy policy.

### References(s):

- Phi-2 on [HuggingFace](https://huggingface.co/microsoft/phi-2) 
- Microsoft Research [blog post](https://www.microsoft.com/en-us/research/blog/phi-2-the-surprising-power-of-small-language-models) about Phi-2 

### Model Architecture:

**Architecture Type:** Transformer 
**Network Architecture:** Phi 

### Input:

**Input Type(s):** Text 
**Input Format:** Text 
**Input Parameters:** Temperature, TopP 
**Other Properties Related to Input:** None 

### Output: 

**Output Type(s):** Text 
**Output Format:** Text 
**Output Parameters:** Max output tokens 
**Other Properties Related to Output:** None 

### Inference:

**Engine:** [Triton](https://developer.nvidia.com/triton-inference-server)

phi-2

Phi-2 is a 2.7 billion parameter language model developed by Microsoft Research.  The phi-2 model is best suited for prompts using the QA format, the chat format, and the code format

The list of completion choices the model generated for the input prompt.

Choices

Usage statistics for the completion request.

ChatCompletion

ChatCompletionChunk

A word or list of words not to use. The words are case sensitive.

The maximum number of tokens to generate in any given call. Note that the model is not aware of this value, and generation will simply stop at the number of tokens specified.

Max Tokens

A list of messages comprising the conversation so far. The roles of the messages must be alternating between `user` and `assistant`. The last input message should have role `user`. A message with the the `system` role is optional, and must be the very first message if it is present.

Messages

If specified, our system will make a best effort to sample deterministically, such that repeated requests with the same seed and parameters should return the same result.

Seed

A string or a list of strings where the API will stop generating further tokens. The returned text will not contain the stop sequence.

Stop

If set, partial message deltas will be sent. Tokens will be sent as data-only server-sent events (SSE) as they become available (JSON responses are prefixed by `data: `), with the stream terminated by a `data: [DONE]` message.

Stream

The sampling temperature to use for text generation. The higher the temperature value is, the less deterministic the output text will be. It is not recommended to modify both temperature and top_p in the same call.

Temperature

The top-p sampling mass used for text generation. The top-p value determines the probability mass that is sampled at sampling time. For example, if top_p = 0.2, only the most likely tokens (summing to 0.2 cumulative probability) will be sampled. It is not recommended to modify both temperature and top_p in the same call.

Top P

ChatRequest

The reason the model stopped generating tokens. This will be `stop` if the model hit a natural stop point or a provided stop sequence, or `length` if the maximum number of tokens specified in the request was reached.

Finish Reason

The index of the choice in the list of choices (always 0).

Index

A chat completion message generated by the model.

Choice

A chat completion delta generated by streamed model responses.

The reason the model stopped generating tokens. This will be `stop` if the model hit a natural stop point or a provided stop sequence, or `length` if the maximum number of tokens specified in the request was reached. Will be `null` if the model has not finished generating.

ChoiceChunk

Function instance used to invoke the request

InvokeError

Content

Role

Message

Number of tokens in the generated completion.

Completion Tokens

Prompt Tokens

Total number of tokens used in the request (prompt + completion).

Total Tokens

Usage

NVCF function

Result is pending. Client should poll using the requestId.


Validation failed, provided entity could not be processed.

requestId

Gets the result of an earlier function invocation request that returned a status of 202.