Model reference · open weights
jina-clip is an open-weight embedding model from jinaai, listed in the AxForge catalogue. AxForge can bring it up on EU-owned hardware for you on request — with the licence handled where one is required.
About
This model is based on the paper jina-clip-v2: Multilingual Multimodal Embeddings for Text and Images. Quick Start Blog | Technical Report | Azure | AWS SageMaker | Google Cloud Platform | API Intended Usage & Model Info jina-clip-v2 is a general-purpose multilingual multimodal embedding model for text & images. Multimodal embeddings enable searching and understanding data across different modalities through a coherent representation. They serve as the backbone of neural information retrieval and multimodal GenAI applications. Built upon jina-clip-v1 and our recently released jina-embeddings-v3, jina-clip-v2 features several significant improvements: Improved Performance: v2 shows a 3% performance improvement over v1 in both text-image and text-text retrieval tasks. Similar to v1, v2's text encoder can serve as an effective multilingual long-context dense retriever. It performs on par with our frontier model jina-embeddings-v3 (currently the best multilingual embeddings under 1B parameters on MTEB). Multilingual Support: Using the same backbone as jina-embeddings-v3 for the text tower, jina-clip-v2 supports 89 languages for multilingual-image retrieval, showing up to 4% improvement compared to nllb-clip-large-siglip on multilingual image retrieval tasks. Higher Image Resolution: v2 now supports 512x512 input image resolution, a significant increase from v1's 224x224. This higher resolution enables better processing of detailed images, improved feature extraction, and more accurate recognition of fine-grained visual elements. Matryoshka Representations: v2 allows users to truncate the output dimensions of both text and image embeddings from 1024 down to 64, reducing storage and processing overhead while maintaining strong performance. Measuring 0.9B parameters, jina-clip-v2 combines two powerful encoders: the text encoder Jina-XLM-RoBERTa (the backbone of jina-embeddings-v3) and the vision encoder EVA02-L14 (an efficient vision Transformer developed by BAAI). These encoders are jointly trained to create aligned representations of images and text. CLIP-like models have established themselves as the backbone for general-purpose multimodal applications. With jina-cl
Summarised from the published model card. Read the full card on the HuggingFace links below.
Specifications
| Maker | jinaai |
|---|---|
| Type | Embedding models |
| Parameters (lead) | 865M |
| Variants | 2 |
| Runs with | transformers |
| Based on | jinaai/xlm-roberta-flash-implementation |
| Released | 2024-10-08 |
| Popularity | 104k downloads / month |
| Likes | 342 |
| Licence | Commercial licence needed |
How it works
Variants
Open weights ship in several sizes and precisions. One page, all the variants — pick the one that fits your GPU. VRAM figures are estimates from model size.
Using it via the API
Once AxForge deploys jina-clip for you, it answers on the OpenAI-compatible API — the same base URL and keys as every other model. (jina-clip below is illustrative; you get the exact model name on deployment.)
$ curl -sS https://api.axforge.ai/v1/embeddings \
-H "Authorization: Bearer $AXFORGE_API_KEY" \
-H "Content-Type: application/json" \
-d '{"model":"jina-clip","input":"text to embed"}'
Licence
The weights are open but cc-by-nc-4.0 needs a commercial agreement for business use. AxForge can arrange that licence and host the model for you — you pay AxForge, we settle with the model’s maker. Ask us for a quote. Read the licence ↗
Explore