Qwen3.8-Flash-Next

About

What Qwen3.8-Flash-Next is

How to Run Qwen3.8-Flash-Next Locally Running a 176B model on a 64 GB MacBook Qwen3.8-Flash-Next has 177B parameters. Our 85 GB quant runs on an M5 Max with 64 GB of memory, with vision, at 36 tok/s. That is not a typo: the file is larger than the machine's entire RAM. It works because 39 GB of that file never enters memory at all. Why this is possible 51B of the model's 177B parameters are not weights in the usual sense. They are an n-gram lookup table. The model hashes the last three tokens, and that hash points at 16 rows of 160 values each. Roughly 2.7 KB per token, read once per forward pass, out of a 39 GB table. That is a 1-in-13-million read ratio, at a deterministic address. At 36 tok/s it comes to about 3 MB/s of random reads, and NVMe answers in under 100 µs against a 28 ms per-token budget. Common n-grams stay in page cache anyway. Compare that with the experts: they touch about 6B parameters per token, gigabytes of traffic, and would be hopeless from disk. That is why ordinary offloading fails when you run out of memory, and why this table is different. [!IMPORTANT] On Apple Silicon this only works if the table sits in its own GGUF shard. llama.cpp hands Metal the entire mmap'd region of any shard containing GPU tensors, so a table interleaved with weights gets wired along with them. The model then asks for more memory than the machine has, and the first decode dies with kIOGPUCommandBufferCallbackErrorOutOfMemory. Every quant here is split so that shard 2 holds nothing but the table. The quants AD-4.27bpw is the one to take. It matches the 5.00bpw build within measurement error while being 17.6 GB smaller, and leaves more headroom for context. The 3.84bpw build is the one in the video; it is kept for machines where every gigabyte of RAM counts. "In memory" is what the GPU actually holds. The n-gram table is excluded because it stays on SSD. Everything was measured against one reference, one corpus, one machine: the BF16 model's own logits over a held-out neutral set, 87 chunks at 4096 context, BF16 PPL 4.0445 ± 0.0216. Other publishers' files were downloaded and re-measured here rather than having their numbers copied, because figures taken against

Summarised from the published model card. Read the full card on the HuggingFace links below.

Specifications

What it is

How it works

How language models work

Variants

Sizes & precisions

Open weights ship in several sizes and precisions. One page, all the variants — pick the one that fits your GPU. VRAM figures are estimates from model size.

Using it via the API

Call it like any OpenAI endpoint

Once AxForge deploys atomicchat-qwen3-8-flash-next for you, it answers on the OpenAI-compatible API — the same base URL and keys as every other model. (atomicchat-qwen3-8-flash-next below is illustrative; you get the exact model name on deployment.)

$ curl -sS https://api.axforge.ai/v1/chat/completions \
  -H "Authorization: Bearer $AXFORGE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"atomicchat-qwen3-8-flash-next","messages":[{"role":"user","content":"Hello"}]}'

Details

Languages, data & research

Licence

Commercial licence needed

Sources

Weights & code

Explore

More language models