Model reference · open weights
Qwen3-VL-Embedding is an open-weight embedding model from RamManavalan, listed in the AxForge catalogue. AxForge can bring it up on EU-owned hardware for you on request — with the licence handled where one is required.
About
Qwen3-VL-Embedding-8B-FP8 This is an FP8 quantized version of Qwen/Qwen3-VL-Embedding-8B, optimized for efficient inference with vLLM. Model Overview Highlights - Multimodal Versatility: Handles text, images, screenshots, and video inputs - Efficient Inference: ~45% memory reduction with minimal accuracy loss - vLLM Compatible: Works with vLLM's pooling runner for high-throughput embedding - No Calibration Required: Uses FP8DYNAMIC scheme (data-free quantization) Quantization Details - Scheme: FP8DYNAMIC - Weights: FP8E4M3 (per-channel quantization) - Activations: Dynamic per-token quantization at runtime - Tool: llm-compressor - Calibration: None required (data-free quantization) Hardware Requirements - GPU: NVIDIA GPU with FP8 support (compute capability = 8.9) - Blackwell: RTX 5090, RTX 5080 - Ada Lovelace: RTX 4090, RTX 4080 - Hopper: H100, H200 - VRAM: ~10GB minimum for inference Usage With vLLM (=0.14.0) (Recommended) With vLLM (=0.14.0) Server With Transformers Using the Helper Class This repository includes a helper class for easier embedding extraction: Benchmark Results The base model achieves state-of-the-art performance on multimodal benchmarks: FP8 quantization typically preserves 95% of the original model's accuracy. Creation This model was quantized using llm-compressor: Citation If you use this model, please cite the original Qwen3-VL-Embedding paper: License Apache 2.0 (same as base model) Acknowledgments - Qwen Team for the original Qwen3-VL-Embedding model - vLLM Team for the inference engine - Neural Magic for llm-compressor
Summarised from the published model card. Read the full card on the HuggingFace links below.
Specifications
| Maker | RamManavalan |
|---|---|
| Type | Embedding models |
| Parameters (lead) | 8.8B |
| Variants | 1 |
| Runs with | transformers |
| Based on | Qwen/Qwen3-VL-Embedding-8B |
| Released | 2026-01-22 |
| Popularity | 357k downloads / month |
| Likes | 4 |
| Licence | Open weights |
How it works
Variants
Open weights ship in several sizes and precisions. One page, all the variants — pick the one that fits your GPU. VRAM figures are estimates from model size.
| Variant | Params | Precision | VRAM | Fits 16 GB | Weights |
|---|---|---|---|---|---|
| Qwen3-VL-Embedding-8B-FP8 | 8.8B | FP8 | ~10.1 GB | ✓ | Weights ↗ |
Using it via the API
Once AxForge deploys rammanavalan-qwen3-vl-embedding for you, it answers on the OpenAI-compatible API — the same base URL and keys as every other model. (rammanavalan-qwen3-vl-embedding below is illustrative; you get the exact model name on deployment.)
$ curl -sS https://api.axforge.ai/v1/embeddings \
-H "Authorization: Bearer $AXFORGE_API_KEY" \
-H "Content-Type: application/json" \
-d '{"model":"rammanavalan-qwen3-vl-embedding","input":"text to embed"}'
Licence
Open weights under apache-2.0 — commercial use is permitted. Deploy it on AxForge EU hardware on request. Read the licence ↗
Explore