Prism-Qwen3.5-Reranker

Released models

Five checkpoints are released on the Hugging Face Hub. Four are fine-tuned from the Qwen3.5 backbone; one (-4B-exp) is an experimental extension built on top of Qwen3-Reranker-4B, demonstrating that the same recipe transfers to an existing LLM-based reranker without losing ranking quality.

Why this model?

In agentic / RAG pipelines, a relevance score is rarely the end goal. After deciding a document is relevant, the agent still has to read it, denoise it, and decide what to do next. Prism-Reranker folds that work into the reranker itself:

• Relevance score — s(q, d) = σ(ℓ_yes − ℓ_no) ∈ (0, 1). Calibrated, ranking-ready.
• `` — one sentence stating every core point the document contributes to the query. Useful for the agent to plan its next step without re-reading the doc.
• **** — a self-contained, faithfully-rephrased rewrite of the query-relevant content. Drops irrelevant background, preserves verbatim proper nouns / numbers / dates / code / URLs. You can feed directly to a downstream LLM and skip the raw document — saving context tokens and removing web-noise.

If the document is not relevant, the model outputs no and stops. No contribution/evidence is generated.

Highlights

• Backbones: Qwen3.5 series for the four main sizes, no architectural changes; one extension variant on top of Qwen3-Reranker-4B.
• Context length: training data capped at 10K tokens per example, covering most real-world documents.
• Multilingual: Chinese / English primary; other languages supported but with less coverage.
• Keyword-query robust: agents often emit keyword-style queries instead of well-formed questions. ~30% of training queries were rewritten by an LLM into keyword form, so the model handles both natural and keyword queries.
• Real-world data distribution: in addition to open reranker datasets (MS MARCO, T2Ranking, MIRACL, …), training includes synthetic queries paired with real Tavily / Exa web-search results, matching what an actual agent sees at inference time.
• Length × score balanced: training data was rebalanced so that document length is not a relevance shortcut.
• Training recipe: distillation (point-wise MSE on a strong commercial reranker's scores) + SFT on yes/no + +, supervised by a 5-LLM-as-judge ensemble.

Quickstart

Two ways to call the model. Both produce the same relevance score s(q, d) = σ(ℓ_yes − ℓ_no). Use A when you also want /. Use B when you only need a score and want a drop-in replacement for any other CrossEncoder reranker.

We use one shared example throughout so you can compare the outputs side by side:

QUERY = "What is the boiling point of water at sea level?"
DOCUMENTS = [
    "Water boils at 100 C (212 F) at standard atmospheric pressure (1 atm), "
    "which corresponds to sea-level conditions.",
    "Mount Everest is the highest mountain on Earth, with a peak elevation "
    "of 8,848 meters above sea level.",
]

A. Transformers (full output: score + contribution + evidence)

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

MODEL_PATH = "infgrad/Prism-Qwen3.5-Reranker-4B"  # or any sibling repo above

SYSTEM_PROMPT = (
    "Judge whether the Document meets the requirements based on "
    "the Query and the Instruct provided. "
)

INSTRUCTION = (
    'Judge if the document is relevant to the query. Reply "yes" or "no".\n'
    'On "yes", also emit:\n'
    "One sentence covering every core point the document "
    "contributes to the query, without elaboration.\n"
    "Self-contained rewrite of the query-relevant content. Rules:\n"
    "- Faithful: rephrase only; add or infer nothing.\n"
    "- Self-contained: evidence alone must fully answer the query.\n"
    "- Concise: drop query-irrelevant background.\n"
    "- Verbatim (no translation): proper nouns, terms, abbreviations, "
    "numbers, dates, code, URLs.\n"
    "- Output language: multilingual doc → query's language; else doc's language."
    ""
)

PROMPT_TEMPLATE = (
    "system\n{system}\n"
    "user\n"
    ": {instruction}\n"
    ": {query}\n"
    ": {doc}\n"
    "assistant\n\n\n\n\n"
)

def build_prompt(query: str, doc: str) -> str:
    return PROMPT_TEMPLATE.format(
        system=SYSTEM_PROMPT, instruction=INSTRUCTION, query=query, doc=doc
    )

tokenizer = AutoTokenizer.from_pretrained(MODEL_PATH)
model = AutoModelForCausalLM.from_pretrained(
    MODEL_PATH,
    torch_dtype=torch.bfloat16,
    device_map="cuda",
    attn_implementation="sdpa",
).eval()

yes_id = tokenizer.encode("yes", add_special_tokens=False)[0]
no_id = tokenizer.encode("no", add_special_tokens=False)[0]

@torch.no_grad()
def rerank(query: str, doc: str, max_new_tokens: int = 512):
    prompt = build_prompt(query, doc)