needle2

About

What needle2 is

Needle 2 Needle 2 is an open 45M-parameter model for tool calling, device use and structured extraction. The whole model is a single 14MB binary that runs a full session in 28MB of RAM. It is built on our Simple Attention Network findings, compressed to CQ2-bit with Cactus Quants, and baked into its own engine. On the benchmarks below, Needle 2 trades wins with other small models like FunctionGemma 270M, LFM2.5 230M and Apple FM, at 5x to 70x smaller, and 2 bits against their f16. Needle hits 500 tokens/sec decode speed on a Raspberry Pi 5, between 400-1,500 tokens/sec on VR devices like Meta Quest 3S and Apple Vision Pro, and ranges 300-700 on sub-$200 phones such as the Samsung A-Series. With a peak session RAM around 28MB, Needle reaches microcontrollers like the ESP32-P4; others have reported running it on an ESP32-S3 in about 11MB. - Self-contained: model baked into the binary, no runtime, no downloads, no network. - Runs everywhere: ARM64, x86-64, ARMv7, RISC-V, and WebAssembly, on Apple, Windows, Linux, Android, Raspberry Pi. - Simple contract: tool calls come back as structured data, text in, JSON out; a byte-level grammar compiled from your schemas constrains every token. - Confidence-gated: every response carries a calibrated confidence score from a learned head; set a threshold, act above it, escalate below it. - Tool retrieval: declare a large catalogue and a built-in retrieval head renders only the top five tools per turn, with the grammar constrained to that subset. - Bounded memory: a 256-token sliding window with the tools pinned as KV sinks, so total memory stays near 28MB no matter how long the conversation runs. Source, engine, and training code: github.com/cactus-compute/needle. Simple Attention Network Needle 2 is a Simple Attention Network, our dense small-model recipe: a Hadamard MLP in place of the FFN, GQA attention, engram key-value memory, and multi-lane hyper-connections. See the paper for the design and ablations: arXiv:2607.18363. Each block carries its update rule. Here x̂ is the RMS-normalised flattening of the four residual streams, H the orthonormal Walsh-Hadamard transform (a fixed matrix, applied in n log n time with no weight

Summarised from the published model card. Read the full card on the HuggingFace links below.

Specifications

What it is

How it works

How language models work

Variants

Sizes & precisions

Open weights ship in several sizes and precisions. One page, all the variants — pick the one that fits your GPU. VRAM figures are estimates from model size.

Using it via the API

Call it like any OpenAI endpoint

Once AxForge deploys needle2 for you, it answers on the OpenAI-compatible API — the same base URL and keys as every other model. (needle2 below is illustrative; you get the exact model name on deployment.)

$ curl -sS https://api.axforge.ai/v1/chat/completions \
  -H "Authorization: Bearer $AXFORGE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"needle2","messages":[{"role":"user","content":"Hello"}]}'

Details

Languages, data & research

Papers

Licence

Open weights

Sources

Weights & code

Explore

More language models