Qwen3.8-Ridge

About

What Qwen3.8-Ridge is

Qwen3.8-27B-Ridge-3.7bpw Developed by Empero A Gated-DeltaNet-aware mixed GGUF of official Qwen/Qwen3.8-27B (1d4bf0f2) for llama.cpp, Ollama, LM Studio, jan, KoboldCpp, and other stock GGUF runtimes. This is a quantization of the Qwen3.8-27B checkpoint. Ridge is a probed mix of types written for this architecture: 64 layers = 16 × (3 × GatedDeltaNet → FFN + 1 × GatedAttn → FFN). Generic IQ2XS and UD-IQ2 do not treat GDN state (ssmalpha / ssmbeta) or the GDN mixers as first-class. We fixed that. Nothing was stripped to make the file fit. The native MTP draft head (blk.64 / nextn) stays in the GGUF. Vision is a separate BF16 mmproj. [!Note] This card is about choosing the file and running it. The official capability writeup lives on the base model card. Files The repository is Qwen3.8-27B-Ridge-GGUF. Use the exact filenames below when downloading or passing -m. If you only want text, download the Ridge GGUF. Add the mmproj for image input. What fits on a GPU? These are practical weight-size-based estimates, not a VRAM benchmark. They assume a modest context and leave room for runtime and the KV cache. Image input adds the 0.87 GiB mmproj. The native 262k window and the 1M YaRN extension — make KV the dominant cost and may need offload regardless of weight quant. Measured: Qwen3.8-27B-Ridge-3.7bpw.gguf fully offloaded to a single RTX PRO 6000 Blackwell (96 GB) runs at ~54 tok/s generation, ~130 tok/s prompt (llama.cpp CUDA, -ngl 99, short smoke). One data point on one card, not a sweep — but a 27B at 11.7 GiB is comfortably interactive on a 16–24 GB card at modest context. Recipe Qwen3.8 is a hybrid: three Gated-DeltaNet layers for every full-attention layer. GDN state is disproportionately sensitive to low-bit quantization, so Ridge holds that path high and spends the saved bits by dropping mid-stack FFN. The Gated-DeltaNet state path is Q80. Mixers are Q4K, not IQ2. That is the difference between this file and a flat 2-bit dump of the same model. Built with llama.cpp adb55e5, CUDA, importance matrix on 80 × 512-token chunks (--process-output, wikitext + code). MTP tensors are unused during calibration and have no imatrix — IQ2/IQ3 on blk.64 will abort, so the dra

Summarised from the published model card. Read the full card on the HuggingFace links below.

Specifications

What it is

How it works

How language models work

Variants

Sizes & precisions

Open weights ship in several sizes and precisions. One page, all the variants — pick the one that fits your GPU. VRAM figures are estimates from model size.

Using it via the API

Call it like any OpenAI endpoint

Once AxForge deploys qwen3-8-ridge for you, it answers on the OpenAI-compatible API — the same base URL and keys as every other model. (qwen3-8-ridge below is illustrative; you get the exact model name on deployment.)

$ curl -sS https://api.axforge.ai/v1/chat/completions \
  -H "Authorization: Bearer $AXFORGE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"qwen3-8-ridge","messages":[{"role":"user","content":"Hello"}]}'

Details

Languages, data & research

Licence

Open weights

Sources

Weights & code

Explore

More language models