GLM-5.3-Flash

About

What GLM-5.3-Flash is

GLM-5.3-Flash · NVFP4 320B total · 18B active · natively multimodal · 1M context 598.5 GiB → 181 GiB  ·  −70%  ·  round-trip cosine 0.99665 [](https://huggingface.co/zai-org/GLM-5.3-Flash) [](https://github.com/NVIDIA/TensorRT-Model-Optimizer) [](https://opensource.org/license/mit) Weight-only NVFP4 quantization of Z.ai's GLM-5.3-Flash — the first GLM to combine sparse and linear attention. Quantized by LibertAI · not affiliated with Z.ai / Zhipu ✨ What this is A weight-only NVFP4 (NVFP4-A16) checkpoint. The routed-expert FFN tensors — 97% of the model's parameters — are quantized to NVFP4 (E2M1, with FP8-E4M3 per-16-block scales and an FP32 per-tensor global scale). Everything outlier-sensitive stays in BF16: both attention flavours (all 34 KDA linear-attention layers and the 11 DeepSeek-sparse layers, including the sparse indexer) · the entire vision tower · shared experts · MoE routers · dense-MLP layers · the MTP head · the Manifold-Constrained Hyper-Connection (mHC) tensors · token embeddings · lmhead · all norms Activations stay BF16 — there is no activation quantization. Because the experts dominate the footprint, quantizing only them buys the full 70% while protecting quality where it matters. Leaving the vision tower untouched keeps multimodal behaviour bit-identical to the source. Produced with NVIDIA ModelOpt 0.45.0 through a memory-frugal shard-streaming pass — CPU-only, never more than one shard resident, no calibration data (weight-only NVFP4 derives its scales from the weights themselves). 🚦 Engine support — read this first glm5next is a brand-new architecture (released 2026-08-26) and is not yet in vLLM main (vllm#53906, sglang#36507). Support ships in per-model images. ✅ vLLM on sm121 (GB10) now works, via Libertai/glm53-flash-vllm-gb10. Two independent faults had to be fixed and either one alone leaves the model degenerate, which is why it long looked like a single unexplainable bug. First, no vLLM MLA backend accepted this model's NoPE dimensions on sm121, since the sparse decode path asserts pedim == 64 and GLM-5.3-Flash has qkropeheaddim = 0; that is fixed by a hand-written sparse-MLA CUDA kernel. Second, and not specific

Summarised from the published model card. Read the full card on the HuggingFace links below.

Specifications

What it is

How it works

How language models work

Variants

Sizes & precisions

Open weights ship in several sizes and precisions. One page, all the variants — pick the one that fits your GPU. VRAM figures are estimates from model size.

Using it via the API

Call it like any OpenAI endpoint

Once AxForge deploys libertaidai-glm-5-3-flash for you, it answers on the OpenAI-compatible API — the same base URL and keys as every other model. (libertaidai-glm-5-3-flash below is illustrative; you get the exact model name on deployment.)

$ curl -sS https://api.axforge.ai/v1/chat/completions \
  -H "Authorization: Bearer $AXFORGE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"libertaidai-glm-5-3-flash","messages":[{"role":"user","content":"Hello"}]}'

Details

Languages, data & research

Licence

Open weights

Sources

Weights & code

Explore

More language models