Qwen3-Reranker-seq-cls-vllm

About

What Qwen3-Reranker-seq-cls-vllm is

prudant/Qwen3-Reranker-4B-seq-cls-vllm-fixed-W4A16 Qwen3 4b reranker full vllm adapted 🚀 This is a compressed version of danielchalef/Qwen3-Reranker-4B-seq-cls-vllm-fixed using llm-compressor with the following scheme: W4A16 Serving python3 -m vllm.entrypoints.openai.apiserver --model 'dolfsai/Qwen3-Reranker-4B-seq-cls-vllm-W4A16' --task classify Important: You MUST read the following guide for correct usage of this model here Guide Model Details - Original Model: danielchalef/Qwen3-Reranker-4B-seq-cls-vllm-fixed - Quantization Method: GPTQ - Compression Libraries: llm-compressor - Calibration Dataset: ultrachat200k (512 samples) - Optimized For: Inference with vLLM - License: same as original model

Summarised from the published model card. Read the full card on the HuggingFace links below.

Specifications

What it is

How it works

How embedding models work

Variants

Sizes & precisions

Open weights ship in several sizes and precisions. One page, all the variants — pick the one that fits your GPU. VRAM figures are estimates from model size.

Using it via the API

Call it like any OpenAI endpoint

Once AxForge deploys qwen3-reranker-seq-cls-vllm for you, it answers on the OpenAI-compatible API — the same base URL and keys as every other model. (qwen3-reranker-seq-cls-vllm below is illustrative; you get the exact model name on deployment.)

$ curl -sS https://api.axforge.ai/v1/embeddings \
  -H "Authorization: Bearer $AXFORGE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"qwen3-reranker-seq-cls-vllm","input":"text to embed"}'

Details

Languages, data & research

Licence

Open weights

Sources

Weights & code

Explore

More embedding models