Bonsai

About

What Bonsai is

1-bit Bonsai 27B Full 27B-class reasoning in binary transformer weights — the first 27B-class model to run on a phone ~14.2x smaller than FP16 | ~90% of FP16 intelligence retained | ~11 tok/s on iPhone 17 Pro Max Highlights - ~3.9 GB deployed footprint (down from ~54 GB FP16) — fits within the per-app memory budget of a high-end phone such as the iPhone 17 Pro Max - Retains thinking, reasoning, and agentic behavior deep in the sub-4-bit regime, where conventional low-bit representations collapse — 76.11 average across 15 thinking-mode benchmarks (89.5% of FP16), including math at 91.66 and coding at 81.88 - End-to-end binary language weights across embeddings, attention projections, MLP projections, and LM head, at a true 1.125 bits per weight — no high-precision escape hatches behind a low-bit label; the vision tower ships in compact 4-bit HQQ - 262K-token context on-device, kept practical by the Qwen3.6-27B hybrid-attention backbone (~75% linear attention) and 4-bit KV-cache quantization - First interactive 27B-class generation on a phone: ~11 tok/s on iPhone 17 Pro Max; ~44 tok/s on an Apple M5 Pro laptop - Custom 1-bit hybrid-attention kernels on Apple MLX (Python, Swift) and CUDA — packed weights are consumed directly, never expanded back to FP16 - Ships with a DSpark speculative-decoding drafter layer trained against the Bonsai 27B target — a lossless 1.37x decode speedup on the CUDA serving path - Ternary companion: also available as Ternary Bonsai 27B, the quality-oriented operating point (~7.2 GB, 95% of FP16) for laptops and GPUs Resources - Whitepaper — full methodology, benchmarks, and measurement notes - Demo & examples — serving, benchmarking, and integrating Bonsai - Low-bit kernels: MLX fork (Apple Silicon) · mlx-swift fork (iOS/macOS) · llama.cpp fork (CUDA) - Discord — join the community for support, discussion, and updates Model Overview Weight Representation: Binary g128 Each weight is a single sign bit: 0 maps to −scale, 1 maps to +scale. Every group of 128 weights shares one FP16 scale factor. Effective bits per weight: 1.125 (1 sign bit + 16-bit scale amortized over 128 weights) — an idealized 14.2x reduction vs FP16. This is the most aggr

Summarised from the published model card. Read the full card on the HuggingFace links below.

Specifications

What it is

How it works

How language models work

Variants

Sizes & precisions

Open weights ship in several sizes and precisions. One page, all the variants — pick the one that fits your GPU. VRAM figures are estimates from model size.

Using it via the API

Call it like any OpenAI endpoint

Once AxForge deploys bonsai for you, it answers on the OpenAI-compatible API — the same base URL and keys as every other model. (bonsai below is illustrative; you get the exact model name on deployment.)

$ curl -sS https://api.axforge.ai/v1/chat/completions \
  -H "Authorization: Bearer $AXFORGE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"bonsai","messages":[{"role":"user","content":"Hello"}]}'

Details

Languages, data & research

Licence

Open weights

Sources

Weights & code

Explore

More language models