Phi-2 (TensorRT LLM)

NVIDIA

Model

NVIDIA

Phi-2 (TensorRT LLM)

Phi-2 is a 2.7 billion parameter language model developed by Microsoft Research. The phi-2 model is best suited for prompts using the Question-Answer (QA) format, the chat format, and the code format.

6 Versions

bs1_int4awq_RTX4090_windows

06/13/2024 11:30 PM UTC1.47 GB

TRT Cloud Engine Properties

Key	Value
cpu_arch	x86_64
os	Windows
num_gpus	1
trtllm_version	0.10.0
max_batch_size	1
weightless	False
gpu	RTX4090

bs1_int4awq_RTX4070_windowsSelected

06/13/2024 11:29 PM UTC1.47 GB

TRT Cloud Engine Properties

Key	Value
cpu_arch	x86_64
os	Windows
num_gpus	1
trtllm_version	0.10.0
weight_stripped	False
max_batch_size	1
gpu	RTX4070

bs1_int4awq_RTX4060TI_windows

06/13/2024 11:29 PM UTC1.47 GB

TRT Cloud Engine Properties

Key	Value
cpu_arch	x86_64
os	Windows
num_gpus	1
trtllm_version	0.10.0
weight_stripped	False
max_batch_size	1
gpu	RTX4060TI

bs1_int4awq_RTX3070_windows

06/13/2024 11:28 PM UTC1.47 GB

TRT Cloud Engine Properties

Key	Value
cpu_arch	x86_64
os	Windows
num_gpus	1
trtllm_version	0.10.0
max_batch_size	1
weightless	False
gpu	RTX3070

bs1_fp16_RTX4090_windows

06/13/2024 11:28 PM UTC4.78 GB

TRT Cloud Engine Properties

Key	Value
cpu_arch	x86_64
os	Windows
num_gpus	1
trtllm_version	0.10.0
max_batch_size	1
weightless	False
gpu	RTX4090

bs1_fp16_RTX4070_windows

06/13/2024 11:27 PM UTC4.78 GB

TRT Cloud Engine Properties

Key	Value
cpu_arch	x86_64
os	Windows
num_gpus	1
trtllm_version	0.10.0
weight_stripped	False
max_batch_size	1
gpu	RTX4070