GLM-5.3-Flash-tr3-4bpw

About

What GLM-5.3-Flash-tr3-4bpw is

GLM-5.3-Flash TR3 4bpw — current SM120 runtime This is the uniform-K4 EXL3/TR3 routed-expert checkpoint for GLM-5.3-Flash. The current v84 runtime supports three explicit TP2/EP2/DCP2 profiles on two SM120 GPUs: multimodal DFlash2, language-only DFlash2, and language-only MTP3. All use calibrated NVFP4 MLA KV and CUDA graphs. This is a custom vLLM/B12X build and is not compatible with stock upstream vLLM. Encoder reproducibility closure The repository now contains the complete, hash-verified R10 Python encoder closure used by the EXL3/MCG adapter, including r7encoder/r10codec.py (R10TrellisCodec) and the pinned lineage/encodetr3v31.py numeric core. It is published under reproducibility/r10/ with a per-file SHA-256 manifest and an offline verifier: The bundle is byte-identical to the immutable prior-control source at Hugging Face revision 7c73450f05a151439d0f184f216b1eefcc394a31. It contains the portable Python/numeric source, not a compiled exllamav3ext; that binary must still be built for the target PyTorch, CUDA, and SM ABI and is independently hash-bound by the adapter. See the bundle README for the exact adapter paths, lineage boundary, and licensing. Pick a serving profile The MTP3 option means the model's built-in MTP head only: it does not load or mount the external DFlash checkpoint. Choose DFlash2 when its modest C1 decode gain matters more than resident context/concurrency; choose MTP3 for the normal text-only daily driver. Run the current image The runtime image does not contain either checkpoint. Download/mount this EXL3 model and incoai/GLM-5.3-Flash-DFlash2 separately. The DFlash2 checkpoint is distributed under CC-BY-NC-ND-4.0; review its license before use. Docker Compose: Standalone serve script: The published profile has a 98,304-token request ceiling and allocated 129,473 KV tokens on the qualified pair. Its hybrid Mamba/DFlash rollback layout has room for one full resident request; additional requests queue. C2/C4 rows in the raw benchmark are therefore capacity-limited and are not throughput claims. Language-only profile For text serving, use the language-only profile. It disables the vision tower and uses the built-in MTP3 head by default,

Summarised from the published model card. Read the full card on the HuggingFace links below.

Specifications

What it is

How it works

How language models work

Variants

Sizes & precisions

Open weights ship in several sizes and precisions. One page, all the variants — pick the one that fits your GPU. VRAM figures are estimates from model size.

Benchmarks

Reported results

As published on the model card — the maker's own numbers, not measured by AxForge.

Using it via the API

Call it like any OpenAI endpoint

Once AxForge deploys glm-5-3-flash-tr3-4bpw for you, it answers on the OpenAI-compatible API — the same base URL and keys as every other model. (glm-5-3-flash-tr3-4bpw below is illustrative; you get the exact model name on deployment.)

$ curl -sS https://api.axforge.ai/v1/chat/completions \
  -H "Authorization: Bearer $AXFORGE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"glm-5-3-flash-tr3-4bpw","messages":[{"role":"user","content":"Hello"}]}'

Details

Languages, data & research

Licence

Commercial licence needed

Sources

Weights & code

Explore

More language models