Model reference · open weights
ThinkingCap-Qwen3.8 is an open-weight language model from bottlecapai. ThinkingCap-Qwen3.8-27B-NVFP4 (NVFP4) weighs 20.6 GB; the smallest configuration that runs it is 2× RTX 4060 Ti 16 GB.
What it is
| Released by | bottlecapai |
|---|---|
| Type | Language models |
| Task | Vision + text |
| Parameters (lead) | 16.7B |
| Context | 262,144 tokens |
| Based on | bottlecapai/ThinkingCap-Qwen3.8-27B |
| Released | 2026-09-23 |
| Popularity | 755 downloads / month |
| Weights | 20.6 GB (ThinkingCap-Qwen3.8-27B-NVFP4 (NVFP4), file size) |
| Licence | Its own licence terms |
What it runs on
Weights 20.6 GB (file size) · KV cache 66 MB per 1,000 tokens of context, at 16 bits (vLLM's default for this build; an 8-bit cache halves it) · 308 MB of fixed state per request · runtime overhead from 2.2 GB on a small card · context up to 262,144 tokens.
| Card | Requests at once 8K tokens each | Requests at once 32K tokens each | Longest single request | Counted memory |
|---|---|---|---|---|
| RTX 3060 12 GB … RTX 4060 Ti 16 GB 2 smaller cards | — | — | — | |
| RTX 3090 24 GB FP4 without its speed-up here | — | — | 4K | 23.4 GB |
| RTX 4090 24 GB FP4 without its speed-up here | — | — | 3K | 23.4 GB |
| RTX 5090 32 GB | 9 | 3 | 117K | 31.0 GB |
| L40S 48 GB FP4 without its speed-up here | 25 | 8 | all 256K | 44.0 GB |
| A100 80 GB FP4 without its speed-up here | 65 | 22 | all 256K | 78.2 GB |
| H100 80 GB FP4 without its speed-up here | 58 | 20 | all 256K | 78.1 GB |
| RTX PRO 6000 Blackwell 96 GB | 77 | 26 | all 256K | 93.8 GB |
| DGX Spark (GB10) 128 GB unified | 93 | 32 | all 256K | 107 GB |
| H200 141 GB FP4 without its speed-up here | 129 | 44 | all 256K | 138 GB |
| B200 180 GB | 173 | 59 | all 256K | 176 GB |
| 2× RTX 4060 Ti 16 GB tensor parallel · FP4 without its speed-up here | 6 | 2 | 82K | 15.4 GB a card |
| 2× RTX 4090 24 GB tensor parallel · FP4 without its speed-up here | 25 | 8 | all 256K | 23.4 GB a card |
| 2× RTX 3090 24 GB tensor parallel · FP4 without its speed-up here | 25 | 8 | all 256K | 23.4 GB a card |
| Requests at once | 8K tokens each | 32K tokens each |
|---|---|---|
| 1 | 23.6 GB | 25.2 GB |
| 5 | 27.0 GB | 35.1 GB |
| 8 | 29.5 GB | 42.4 GB |
| 16 | 36.3 GB | 62.1 GB |
| 32 | 49.8 GB | 101 GB |
| 64 | 76.9 GB | 180 GB |
On one card, with vLLM's small-card settings (2,048 tokens a step). Cards of 70 GB and more reserve more per request and more overhead — each row above uses its own card's settings.
Estimates, not measurements, checked against published vLLM startup logs. The weights are the build's file size; the cache is calculated from its config (hybrid: linear attention with full attention every few layers); the overhead is an estimate of vLLM's own memory with that card's default settings. "Requests at once" is how many requests of that length vLLM admits — its reservation at full length, with --max-model-len set to that length; requests that stay shorter fit more. "Longest single request" is the most one request can hold there: below the model's maximum, vLLM starts only with --max-model-len set at or under it. "Counted memory" is vLLM's default 92 % of what CUDA reports for the card (the DGX Spark: about 100 GiB of its shared 128 GB). A tensor-parallel split pools the cards' memory and speeds each token up, at the cost of the link between them; a layer split (llama.cpp) holds more but does not make one request faster. Assumes vLLM 0.10 or later.
From the model card
NVFP4 weight-only (NVFP4A16, compressed-tensors nvfp4-pack-quantized, 16-element FP4 groups with FP8 scales), built by llm-compressor. The two GDN projections vLLM fuses into one GEMM (in_proj_qkv, in_proj_z) share one global scale: vLLM keeps a single scale for the fused GEMM, so separate scales would dequantize one of the two too small. Runs on Hopper (Marlin) and Blackwell.
Built from bottlecapai/ThinkingCap-Qwen3.8-27B (bf16).
Vision tower, MTP head, lm_head and the GDN in_proj_a / in_proj_b projections stay bf16 (recipe.yaml). Serve with vLLM 0.29 (--trust-remote-code not needed).
Paired comparison with the bf16 source on the full quantization plan: both builds answer the same questions with the same seeds — RealWorldQA 765 questions × 2 seeds (images), GPQA-Diamond 198 × 4, MMLU-Pro 1,500 × 1 (a fixed slice), IFBench 300 × 2, AA-LCR 100 × 1 (long-document prompts, graded by Gemma-4-26B-A4B-it with thinking off). Thinking at the chat template's default reasoning effort (xhigh), sampled decoding (temperature 1.0, top_p 0.95, top_k 20, min_p 0.0), 65,536-token generation cap. Both builds served by vLLM 0.29.0 on an H200.
| benchmark (questions × seeds) | accuracy %, bf16 → NVFP4 | Δ accuracy, pp [95% CI] | tokens mean / median / p95, bf16 → NVFP4 | Δ mean tokens [95% CI] | Δ median tokens |
|---|---|---|---|---|---|
| RealWorldQA (765 × 2) | 83.1 → 82.5 | −0.7 [−2.2, +0.9] | 488 / 112 / 1,912 → 461 / 116 / 1,910 | −5.5% [−18.9, +10.1] | +3.6% |
| GPQA-Diamond (198 × 4) | 88.0 → 86.4 | −1.6 [−3.9, +0.6] | 7,115 / 1,031 / 37,459 → 6,618 / 1,122 / 33,885 | −7.0% [−14.7, +1.7] | +8.9% |
| MMLU-Pro (1,500 × 1) | 84.1 → 85.1 | +0.9 [−0.4, +2.3] | 1,436 / 166 / 7,255 → 1,330 / 170 / 7,726 | −7.4% [−18.0, +4.9] | +2.4% |
| IFBench (300 × 2) | 79.7 → 78.0 | −1.7 [−4.7, +1.4] | 4,531 / 1,822 / 20,630 → 4,103 / 1,955 / 15,651 | −9.5% [−15.7, −2.3] | +7.3% |
| AA-LCR (100 × 1) | 81.0 → 78.0 | −3.0 [−10.0, +4.0] | 1,718 / 844 / 4,937 → 1,508 / 920 / 4,043 | −12.2% [−22.1, −1.6] | +9.0% |
Δ accuracy is NVFP4 minus bf16 on the same answers; its interval treats the question as the unit (seeds averaged per question first). Tokens are completion tokens (reasoning plus answer). Bold: interval excludes zero.
Mean tokens fall significantly on IFBench and AA-LCR with no significant accuracy change, but the medians there rise 7–9%: the drop is fewer very long answers, not uniformly shorter ones.
Throughput — one RTX PRO 6000 Blackwell, vLLM 0.29.0 (Marlin FP4 kernel for this build), synthetic prompts of 1,024 tokens with 512 generated (last column: 32,768-token prompts, 128 generated), end-of-sequence ignored, prefix caching off, --max-num-seqs 64. Aggregate output tokens/s, median time to first token in ms in parentheses. With 32k-token prompts this build is no faster than bf16 and its first token comes later.
| build | 1 request | 16 concurrent | 64 concurrent | 4 concurrent, 32k-token prompts |
|---|---|---|---|---|
| bf16 | 26.2 (158) | 341 (1,867) | 844 (3,627) | 18.6 (9,678) |
| NVFP4 | 68.0 (176) | 666 (2,404) | 1,149 (4,105) | 18.9 (13,670) |
Decode speed and MTP self-speculative decoding (MMLU-Pro) — 32 questions × 1 seed, vLLM 0.26 on one H200, 16 concurrent requests
| config | median tokens | tok/s | s / task | MTP speedup | accept_len (max 4) |
|---|---|---|---|---|---|
| Qwen3.8-27B base · standard | 484 | 51.3 | 8.9 | 1.00× | — |
| Qwen3.8-27B base · MTP | 502 | 91.0 | 4.2 | 1.77× | 2.59 |
| ThinkingCap-Qwen3.8-27B bf16 · standard | 232 | 50.5 | 3.7 | 1.00× | — |
| ThinkingCap-Qwen3.8-27B bf16 · MTP | 216 | 85.7 | 2.0 | 1.70× | 2.60 |
| NVFP4 · standard | 216 | 61.0 | 2.6 | 1.00× | — |
| NVFP4 · MTP | 215 | 74.9 | 1.7 | 1.23× | 2.54 |
Need even more efficiency? The open release is production-ready. Our enterprise versions go further — fewer thinking tokens still, tuned to your workload, at matched accuracy on your own tasks. Built for AI labs, inference providers and enterprises running models at scale. Deployed on your infrastructure, or in the cloud and region you choose. Talk to our team
ThinkingCap: PolyForm Small Business 1.0.0 + BottleCap personal-use grant (see LICENSE).
Upstream Qwen materials: Apache-2.0 (see NOTICE).
Commercial license: contact BottleCap AI.
If you use this model, please cite:
@misc{ThinkingCap-Qwen3.8-27B,
title = {bottlecapai/ThinkingCap-Qwen3.8-27B},
author = {Osusky, Adam and Lindauer, Jan and Jirkovsky, Adam and Mihal, Filip and Platek, Ondrej and Herel, David and Ihnatchenko, Luka and Bartek, Vojtech and Jirak, Jiri and Kubista, Daniel and Krus, Frantisek and Mikolov, Tomas},
year = {2026},
}
Quoted from the model card on Hugging Face — the full card is behind the Hugging Face link above.