Model reference · open weights
Whittle-Next is an open-weight language model from logic65. AxForge deploys and operates it for you on dedicated EU-owned hardware — with the licence handled where one is required.
Available as managed deployment — configured and operated for you on dedicated EU hardware, quoted per deployment.
What it is
| Released by | logic65 |
|---|---|
| Type | Language models |
| Task | Text gen · MoE |
| Parameters (lead) | 27.6B |
| Context | 256k tokens |
| Runs with | llama.cpp |
| Based on | logic65/Whittle-Next-26B-A3B |
| Released | 2026-09-03 |
| Popularity | 1k downloads / month |
| Licence | Open weights |
About
☕ Out of budget — support this work
Whittle is built by one person on a grocery budget and rented GPU hours, and the compute budget is now spent. The full distillation this model needs (the plan is in Next step below) is the step we cannot currently pay for. If this research is useful to you, or you want to see it finished: ko-fi.com/davida81328. Every hour of GPU time goes straight into the next checkpoint, and every checkpoint, table and log lands in this repo.
What this is, and is not. This is not a finished, general-purpose model. It is a working starting point for anyone who wants to build on the Qwen4-Next (
qwen4_exp) architecture — a body that already loads and runs on stock llama.cpp, with hyper-connections, a hashed n-gram memory and per-layer-embedding tensors wired in, plus every piece needed to keep training it (frozen body, trainable checkpoint, table, hash contract, trainer, exporter). Treat the numbers as a baseline to improve on, not a product.
The successor to Whittle-Next-26B-A3B: the same Qwen3.8-Flash-Next-format
(qwen4_exp) body built from Qwen3.6-35B-A3B, a 2B-row hashed n-gram memory, hyper-connection streams that are genuinely
different from each other, and a distillation lineage from Qwen3.8-27B. Four phases got it here. v1–v3 (3–4 Sep) built the body,
made the memory load-bearing and distilled reasoning to completion. v4 (7–10 Sep, the weights at the root of this repo) replaced the
memory's contents with an exact transfer of Qwen3.8-Flash-Next's own n-gram table, taught the layers after the injection point to read
it, and then ran three nights of expert-only distillation on complete thinking traces, which took the code-review runaways from 6 in 40
to 0. Runs on stock llama.cpp, no patches.
Read the caveats section before using it as anything but a research artefact.
| base 25B | Next-26B | Next-27B v3 (last full 200-question eval) | |
|---|---|---|---|
| total parameters | 25.1B | 26.1B | 27.1B (25.1B + 2.0B memory) |
| active per token | ~3B | ~3B | ~3B |
| GSM8K 200, 512-token cap, thinking off | 86.5% | 87.0% (4 truncated) | 86.5% (173/200, 1 truncated) |
| GSM8K 200, 1024-token cap, thinking off | – | 86.5% (0 truncated) | 86.0% (172/200, 0 truncated) |
| GSM8K 200, 2048-token cap, thinking on | – | – | 86.5% (173/200, 3 truncated) |
| stop probe (12 open replies, 400 cap) | – | 10/12 | 12/12, max 4-gram repetition 0.055 |
| code probes (fib / bash / LRU) | – | – | all three correct; 2 of 3 fenced |
| held-out CE, chat / corpus | – | 1.182 / 2.052 | 1.219 / 2.084 (teacher on the same corpus rows: 1.826) |
All numbers in this table are from the v3 Q8_0 GGUF on stock llama.cpp with the serving sampler below; every GSM8K reply and probe log is
in eval/v3/. v4 has so far been measured on a 40-prompt thinking-on probe (below), not on the full 200-question set - that eval is the
next thing to run, and until it lands treat v4's maths as "v3-level, unverified at scale".
The three phases are kept here because each one measured something that changed the plan. Every checkpoint, table copy and log
of every phase, including the ones that were not shipped, is in train/ on this repo and on the 26B repo.
v1 measured 84.5% / 86.0% on GSM8K at 512 / 1024 (2 and 1 truncated), 9/12 on the stop probe, 1 of 3 code probes fully right.
Tried and not shipped (logs under train/27b* on the 26B repo): a six-hour table-only night (introduced maths loops, 15/200
runaways), a code-only phase, and two LoRA-on-everything attempts at 1e-4 to 2e-4 that drifted on both held-out sets within 500
steps. The v1 GGUF is kept on this repo for comparison.
v1's 2B-row table had never lowered a held-out number. v2 made it measurably load-bearing and, for the first time, let something downstream of the table learn to read it.
Why the table was inert (measured, not guessed). llama.cpp's PLE path normalises value × gate after gating, so the
learned gate is a no-op at inference: every written row injects at unit scale and only an exactly-zero row injects nothing.
There is no confidence channel, so a table trained to write everywhere writes noise everywhere. A pruning sweep on the v1 table
confirmed the corollary: all of its small benefit lived in a few thousand hot rows.
Sparse commit. v2 trains the table in the memorisation regime it can serve: a fixed 1M-token corpus slice, rows committed only after being visited 6 times (shadow table + visit counts), decay on committed rows, no pruning of the shipped v1 rows.
Adapting the readers. LoRA r8 on the expert and attention linears of layers 2–12 (the layers right after the injection point), lr 3e-5, with a self-anchor term (SelfAug, arXiv 2509.03934): KL(original student ‖ adap
From the published model card. Full card on the HuggingFace links in the sidebar.
Using it via the API
Once AxForge deploys whittle-next for you, it answers on the OpenAI-compatible API — the same base URL and keys as every other model. (whittle-next below is illustrative; you get the exact model name on deployment.)
$ curl -sS https://api.axforge.ai/v1/chat/completions \
-H "Authorization: Bearer $AXFORGE_API_KEY" \
-H "Content-Type: application/json" \
-d '{"model":"whittle-next","messages":[{"role":"user","content":"Hello"}]}'
Create an account — your API key is available in the console. 3M free tokens every 30 days with every new account.