Qwen3.8-RTX5090

About

What Qwen3.8-RTX5090 is

Qwen3.8-27B-NVFP4-RTX5090 GeForce RTX 5090–specific NVFP4 checkpoint of Qwen/Qwen3.8-27B, quantized with NVIDIA Model Optimizer. Built so a 32 GB RTX 5090 can serve the full native 262,144-token context and decode faster than Unsloth’s NVFP4 on the same GPU, with accuracy almost maintained. ### 🚀 Now 2.04× faster with the matching DSpark drafter A speculative drafter trained and NVFP4-quantized specifically against this checkpoint now ships: Qwen3.8-27B-DSpark-NVFP4 — 88.5 → 180.3 tok/s, 1.41 GB, outputs unchanged. It beats the built-in MTP head by 13.8% while using a quarter of its memory. See the numbers. Blackwell tensor cores only. Hopper can load the files but cannot run NVFP4. Why this checkpoint on RTX 5090 Same GPU (RTX 5090 32 GB). Speed: temperature=0, thinking off. Accuracy: temperature=1.0, thinking on, same 20 items per task. This checkpoint: vLLM 0.27.1, util 0.97. Unsloth speed numbers: SGLang 0.5.18 (NEXTN/MTP does not fit usefully on 32 GB). Unsloth accuracy: same vLLM 0.27.1 stack. Unsloth is a different recipe (compressed-tensors mixed NVFP4/FP8, larger weights, MTP draft). On 32 GB that extra size eats the KV pool, so it cannot hold native 256k and decode is about half of this ModelOpt export. Accuracy vs Unsloth NVFP4 Quality is almost maintained on a matched 20-question smoke of GPQA Diamond, AIME 2025, and MMLU-Pro. Overall 45/60 (75%) for both checkpoints. MMLU-Pro tied; GPQA and AIME split by one item each. Method (same for both). RTX 5090, vLLM 0.27.1, thinking on, temperature=1.0, Qwen sampling (topp=0.95, topk=20), --max-model-len 65536, concurrency 4. Same 20 random items per task (sample-seed=20260815, AIME 1 sample each). Datasets: OpenAI simple-evals GPQA Diamond CSV (HF GPQA is gated), MathArena/aime2025, TIGER-Lab/MMLU-Pro 5-shot same-category. Caveats. This is a 20-item smoke, not a full-test ranking. Several misses on both sides were length truncations (thinking filled the 24k/32k generation cap; extract None), not clean wrong answers. Do not treat these as published GPQA / AIME / MMLU-Pro scores. Next weights. Calibration and export are still being tightened. The next drop of this repo is intended to be more accurate and fast

Summarised from the published model card. Read the full card on the HuggingFace links below.

Specifications

What it is

How it works

How language models work

Variants

Sizes & precisions

Open weights ship in several sizes and precisions. One page, all the variants — pick the one that fits your GPU. VRAM figures are estimates from model size.

Using it via the API

Call it like any OpenAI endpoint

Once AxForge deploys qwen3-8-rtx5090 for you, it answers on the OpenAI-compatible API — the same base URL and keys as every other model. (qwen3-8-rtx5090 below is illustrative; you get the exact model name on deployment.)

$ curl -sS https://api.axforge.ai/v1/chat/completions \
  -H "Authorization: Bearer $AXFORGE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"qwen3-8-rtx5090","messages":[{"role":"user","content":"Hello"}]}'

Details

Languages, data & research

Licence

Open weights

Sources

Weights & code

Explore

More language models