Qwen3.8-Flash-Next-REAP-288

About

What Qwen3.8-Flash-Next-REAP-288 is

Qwen3.8-Flash-Next REAP-288 (MLX, 4-bit) Qwen3.8-Flash-Next with 288 of 512 experts per MoE layer, pruned with REAP saliency calibrated on the quantized weights, on the machine that serves them. It is a 180B-class MoE that runs on a 128 GB Mac for Claude Code-style agentic workloads and scores 91.5% on HumanEval. Decode is runtime-dependent: ~28 tok/s on stock mlx-vlm today, and 37 to 65 tok/s on the pmlx engine that ships with the tiered release (see "Decode speed" for the full breakdown). It supports MTP speculative decoding out of the box. Resident memory is ~68 GB as shipped, or ~39 GB when the n-gram table is streamed from NVMe (see "Running it"). - 180B-parameter class: 125B main model, 51B n-gram embedding table, 48 layers alternating Gated DeltaNet and Qwen sparse attention, each with a 288-expert MoE routing top-10 - Runs on a 128 GB Mac at ~68 GB resident on mlx-vlm, or ~39 GB with the n-gram table streamed from NVMe on omlx - Multimodal weights (vision tower) are intact but only text quality has been evaluated Running it For the ~39 GB streamed footprint, run under omlx v0.6.4 or newer. omlx offloads the n-gram table to NVMe automatically when the full model would not fit under the memory ceiling, so the build serves in about 39 GB resident. The server is OpenAI-compatible at http://localhost:8000/v1. mlx-vlm runs the same build today at about 68 GB resident, with the n-gram table in memory. It streams to about 39 GB once PR #2045 (external PLE storage) lands in a release, since this repo already ships the ple-store.json manifest and plestorage config that path reads. mlx-vlm needs qwen4exp MTP support (git main after 2026-08-27, or any release that includes it): Speculative decoding with the model's own MTP head, using the companion drafter sh0wie/Qwen3.8-Flash-Next-MTP-Drafter-MLX-bf16: A note on speculative speed: the drafter's acceptance rate is healthy (~44-68% depending on sampling), but the net speedup depends on how cheaply your hardware runs the verification pass. M5-class GPUs report 1.5-2.6x; on M4 it is roughly break-even. Quality is unaffected either way, since the target model verifies every drafted token. Why 288 experts We measured an

Summarised from the published model card. Read the full card on the HuggingFace links below.

Specifications

What it is

How it works

How language models work

Variants

Sizes & precisions

Open weights ship in several sizes and precisions. One page, all the variants — pick the one that fits your GPU. VRAM figures are estimates from model size.

Using it via the API

Call it like any OpenAI endpoint

Once AxForge deploys qwen3-8-flash-next-reap-288 for you, it answers on the OpenAI-compatible API — the same base URL and keys as every other model. (qwen3-8-flash-next-reap-288 below is illustrative; you get the exact model name on deployment.)

$ curl -sS https://api.axforge.ai/v1/chat/completions \
  -H "Authorization: Bearer $AXFORGE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"qwen3-8-flash-next-reap-288","messages":[{"role":"user","content":"Hello"}]}'

Details

Languages, data & research

Licence

Commercial licence needed

Sources

Weights & code

Explore

More language models