Qwen3.6

About

What Qwen3.6 is

Qwen3.6-35B-A3B-NVFP4 NVFP4 Quantized RedHatAI/Qwen3.6-35B-A3B-NVFP4 This is a preliminary version (and subject to change) of NVFP4 quantized Qwen/Qwen3.6-35B-A3B model. The model has both weights and activations quantized to NVFP4 format with vllm-project/llm-compressor. It is compatible and tested against vllm main. Deploy it with: vllm serve RedHatAI/Qwen3.6-35B-A3B-NVFP4 --reasoning-parser qwen3 --moebackend flashinfercutlass. If you have hardware with more compute than memory bandwidth, you may prefer this MoE variant for performance reasons. Creation Script: Run this script with LLM Compressor main and latest transformers. Evaluation This model was evaluated on GSM8K-Platinum, MMLU-Pro, IFEval, Math 500, GPQA Diamond, AIME 25, and LiveCodeBench v6 using lm-evaluation-harness and lighteval, served with vLLM using --language-model-only. Accuracy Reproduction The results were obtained using the following commands: The model was served with vLLM using the following command: Each benchmark was run 3 times with different seeds (42, 1234, 4158), except AIME 25 which used 8 seeds (42, 1234, 4158, 5322, 1356, 9843, 3344, 5678). Scores are averaged across all seeds. lm-eval benchmarks GSM8K-Platinum (0-shot) IFEval (0-shot) MMLU-Pro (0-shot) lighteval benchmarks litellmconfig.yaml: Math 500, GPQA Diamond, LiveCodeBench v6 (0-shot) AIME 25 (0-shot) BFCLv4 BFCL requires the model to be registered in the leaderboard codebase before running evaluation. Step 1 — Register the model in bfcleval/constants/modelconfig.py Add the following entry to apiinferencemodelmap: Step 2 — Add the key to bfcleval/constants/supportedmodels.py Add "Qwen3.6-35B-A3B-NVFP4" to the SUPPORTEDMODELS list. Step 3 — Start the vLLM server (use the command at the top of this section; the --served-model-name flag ensures BFCL can find the model by its registered slug). Step 4 — Generate responses and evaluate

Summarised from the published model card. Read the full card on the HuggingFace links below.

Specifications

What it is

How it works

How language models work

Variants

Sizes & precisions

Open weights ship in several sizes and precisions. One page, all the variants — pick the one that fits your GPU. VRAM figures are estimates from model size.

Using it via the API

Call it like any OpenAI endpoint

Once AxForge deploys redhatai-qwen3-6 for you, it answers on the OpenAI-compatible API — the same base URL and keys as every other model. (redhatai-qwen3-6 below is illustrative; you get the exact model name on deployment.)

$ curl -sS https://api.axforge.ai/v1/chat/completions \
  -H "Authorization: Bearer $AXFORGE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"redhatai-qwen3-6","messages":[{"role":"user","content":"Hello"}]}'

Details

Languages, data & research

Licence

Open weights

Sources

Weights & code

Explore

More language models