## Model Overview

### Description:

CLIP (Contrastive Language–Image Pretraining) is a model developed by OpenAI that connects vision and language in a way that enables it to understand and generate a wide array of image and text pairs. It has been trained on a large number of internet text–image pairs and can be applied to a variety of downstream tasks. 

### Terms of use

By accessing this model, you are agreeing to the CLIP terms and conditions of the [license](https://github.com/openai/CLIP/blob/main/LICENSE) and [acceptable use policy](https://github.com/openai/CLIP/blob/main/model-card.md#intended-use)

### References(s):

* [CLIP: Connecting Text and Images](https://cdn.openai.com/papers/Learning_Transferable_Visual_Models_From_Natural_Language_Supervision.pdf) paper 
* [OpenAI's CLIP Model Card](https://github.com/openai/CLIP/blob/main/model-card.md) webpage 

### Model Architecture:

**Architecture Type:** Transformer for text and ViT/ResNet for images 
**Network Architecture:** CLIP 

### Input:

**Input Format:** Text and Red, Green, Blue (RGB) Image and Text 
**Input Parameters:** 2D 
**Other Properties Related to Output:** None 

### Output:

**Output Format:** A number indicating the similarity score between image and text 
**Output Parameters:** 2D 
**Other Properties Related to Output:** None 

### Software Integration:

**Runtime(s):** Not Applicable (N/A) 
**Supported Hardware Platform(s):** Hopper, Ampere/Turing 
**Supported Operating System(s):** Linux 

### Training and Evaluation Dataset:

**Links:**
* [COYO-700M](https://github.com/kakaobrain/coyo-dataset) dataset 
* [LAION-Aesthetics](https://laion.ai/blog/laion-aesthetics) dataset 

**Properties (Quantity, Dataset Descriptions, Sensor(s)):** 
COYO-700M contains 747M image-text pairs. LAION-Aesthetics includes several collections of subsets from LAION 5B with high visual quality. 

**Dataset License:**
* [CC-BY-4.0 License](https://github.com/kakaobrain/coyo-dataset/blob/main/LICENSE.cc-by-4.0) for COYO-700M 
* License for LAION-Aesthetics not specified. 

## Inference:

**Engine:** [TensorRT](https://developer.nvidia.com/tensorrt), [Triton](https://developer.nvidia.com/triton-inference-server) 
**Test Hardware:** Other (Not Listed)

clip

The CLIP (Contrastive Language-Image Pretraining) model combines vision and language using contrastive learning. It understands images and text together, enabling tasks like image classification and object detection.