Qwen3.8-QUASAR

About

What Qwen3.8-QUASAR is

Qwen3.8-27B NVFP4, trained with QUASAR QUASAR-QAT/Qwen3.8-27B-QUASAR-NVFP4 is a 4-bit NVFP4 version of Qwen/Qwen3.8-27B, produced with QUASAR, a quantization-aware training (QAT) method. Use it exactly as you would a post-training-quantized (PTQ) build of Qwen3.8-27B: it is a drop-in, lower-precision replacement for the original model, not a fine-tune. The quantized weights are trained by distilling directly from the frozen BF16 original as the teacher, on the teacher's own output distribution, so the model stays anchored to the original's behavior instead of drifting away from it. The difference from PTQ is only that the weights are learned under quantization rather than rounded after the fact, which recovers more of the original quality. 📄 Paper: [QUASAR: Lowering the Loss Floor of Quantization-Aware Training with Loss-Aware Reconstruction](https://arxiv.org/abs/2608.13966) This model uses the most aggressive quantization settings available for Qwen3.8-27B: every linear layer in every transformer layer is NVFP4 (W4A4) — self-attention, gated delta-net, and MLP alike, 496 of 496. Usually for NVFP4 quantization of this model, attention and gated delta-net layers are kept in higher precision (FP8 or BF16), as the model quality can collapse when they are quantized to NVFP4. However, QUASAR preserves quality similar to the original model even when all linear layers (attention, gated delta net, MLPs) are quantized to NVFP4, which makes this the smallest available NVFP4 checkpoint of Qwen3.8-27B and one of the highest quality. How to run Compatible with vLLM, with no conversion step: On a 32 GB card such as an RTX 5090, lower the context to --max-model-len 65536. Requires an NVIDIA GPU with FP4 support (Blackwell, compute capability 10.0+). Quality and size comparison We compare against the two other public NVFP4 builds of this model, both of which reach their size by leaving much of the network above 4 bits: unsloth/Qwen3.8-27B-NVFP4 (23.4 GB) quantizes only 168 of 496 linears to NVFP4, holding all of self-attention and most of the delta-net at FP8, and Inferact/Qwen3.8-27B-NVFP4 (26.4 GB) quantizes 304, leaving 192 of the 240 delta-net projections in BF16. Training

Summarised from the published model card. Read the full card on the HuggingFace links below.

Specifications

What it is

How it works

How language models work

Variants

Sizes & precisions

Open weights ship in several sizes and precisions. One page, all the variants — pick the one that fits your GPU. VRAM figures are estimates from model size.

Using it via the API

Call it like any OpenAI endpoint

Once AxForge deploys qwen3-8-quasar for you, it answers on the OpenAI-compatible API — the same base URL and keys as every other model. (qwen3-8-quasar below is illustrative; you get the exact model name on deployment.)

$ curl -sS https://api.axforge.ai/v1/chat/completions \
  -H "Authorization: Bearer $AXFORGE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"qwen3-8-quasar","messages":[{"role":"user","content":"Hello"}]}'

Details

Languages, data & research

Papers

Licence

Commercial licence needed

Sources

Weights & code

Explore

More language models