jina-embeddings-omni-small-classification

jina-embeddings-v5-omni-small-classification-GGUF: Classification-Targeted Omni Embedding (Small) — GGUF

ArXiv | Blog

Average score vs. parameter count across image (MIEB-Lite), video (MMEB-V), and audio (MAEB) benchmarks — jina-v5-omni-nano and jina-v5-omni-small define the open-weight frontier (Table 1 in the ArXiv report).

Model Overview

GGUF + multimodal-projector build of jinaai/jina-embeddings-v5-omni-small-classification for llama.cpp. Accepts text, images, video, and audio and produces 1024-dim embeddings in the same vector space as the torch reference and as jinaai/jina-embeddings-v5-text-small-classification at the same task — index with text and query with any modality, no reindexing. For a more compact alternative, see jinaai/jina-embeddings-v5-omni-nano-classification-GGUF.

This is the classification-targeted variant of the jina-embeddings-v5-omni-small GGUF family. The umbrella with all GGUF variants and cross-repo benchmarks is jina-ai/jina-embeddings-v5-omni-gguf.

Cross-repo docs: benchmarks (NDCG@5 on NanoBEIR, tokens/sec, peak VRAM, file size), the full per-variant numerical-parity tables, and runtime caveats live in the v5-omni-gguf umbrella.

Via Elastic Inference Service

The fastest way to use v5-omni in production. Elastic Inference Service (EIS) provides managed embedding inference with built-in scaling, so you can generate embeddings directly within your Elastic deployment.

# Retrieve the configuration of the preconfigured omni-small inference endpoint
GET /_inference/embedding/.jina-embeddings-v5-omni-small

# Generate an embedding for a single piece of text using the predefined endpoint
POST _inference/embedding/.jina-embeddings-v5-omni-small
{
  "input": [
    "This is a test"
  ]
}

# Fuse a text description and an image into a single embedding via a multimodal content block
POST _inference/embedding/.jina-embeddings-v5-omni-small
{
  "input": [
    {
      "content": [
        { "type": "text",  "value": "A small blue square" },
        { "type": "image", "format": "base64", "value": "" }
      ]
    }
  ]
}

# Create a custom endpoint that truncates omni-small embeddings to 32 dimensions
PUT _inference/embedding/jina-omni-small-32d
{
  "service": "elastic",
  "service_settings": {
    "model_id": "jina-embeddings-v5-omni-small",
    "dimensions": 32
  }
}

See the Elastic Inference Service documentation for setup details.

Files in this repo

llama.cpp loads --mmproj to enable image / video / audio inputs on top of the text GGUF. The two mmprojs are independent — load whichever modality you need, or pass --mmproj twice to serve both from one process (see "Selective modality loading" below).

Install llama.cpp (with multimodal patches)

This model relies on the Jina v5 omni patches (audio chunked attention, qwen3vl video temporal-pair, encoder combined-decode, etc.) — they are not yet upstream. Build from the feat-v5-omni fork:

git clone https://github.com/jina-ai/llama.cpp.git
cd llama.cpp
git checkout feat-v5-omni
cmake -B build && cmake --build build --config Release -j

For CUDA: pass -DGGML_CUDA=ON to the configure step.

H100 / Hopper note. On Hopper GPUs (H100, H200), set GGML_CUDA_DISABLE_GRAPHS=1 before launching llama-server. Without it, the CUDA-graph capture/replay path crashes with cudaMemcpyAsync … illegal instruction during embedding extraction. CPU, Metal, Vulkan, and pre-Hopper CUDA (e.g. L4, A100) are unaffected.

Quickstart — text via llama-embedding

./build/bin/llama-embedding \
    -hf jinaai/jina-embeddings-v5-omni-small-classification-GGUF:Q4_K_M \
    --pooling last --embd-normalize 2 \
    -p "A cute cat sitting on a mat."

The -hf shortcut downloads + caches the requested quant from this repo on first use. Q4_K_M is the recommended CPU default; Q8_0 for highest fidelity; IQ2_*/IQ1_* for very tight memory budgets.

No prefix convention. Classification text is embedded verbatim — no Query: / Document: prefixes are needed (unlike the retrieval variant). Both sides of any pair go in unprefixed.

No custom pooling o