Model reference · open weights
Naive-N0.5-Flash is an open-weight language model from NaiveAI. Naive-N0.5-Flash (BF16) weighs 618 GB; the smallest configuration that runs it is 4× B200 180 GB.
What it is
| Released by | NaiveAI |
|---|---|
| Type | Language models |
| Task | Text gen |
| Context | 1,048,576 tokens |
| Released | 2026-09-27 |
| Popularity | 605 downloads / month |
| Weights | 618 GB (Naive-N0.5-Flash (BF16), file size) |
| Licence | Open weights |
What it runs on
Weights 618 GB (file size) · KV cache 147 MB per 1,000 tokens of context, at 16 bits (vLLM's default for this build; an 8-bit cache halves it) · runtime overhead from 1.8 GB on a small card · context up to 1,048,576 tokens.
| Card | Requests at once 8K tokens each | Requests at once 32K tokens each | Longest single request | Counted memory |
|---|---|---|---|---|
| RTX 3060 12 GB … 8× A100 80 GB 14 smaller cards | — | — | — | |
| 4× B200 180 GB tensor parallel | 25 | 6 | 202K | 176 GB a card |
| 8× H200 141 GB tensor parallel | 170 | 42 | all 1024K | 138 GB a card |
| Requests at once | 8K tokens each | 32K tokens each |
|---|---|---|
| 1 | 621 GB | 624 GB |
| 5 | 626 GB | 644 GB |
| 8 | 629 GB | 658 GB |
| 16 | 639 GB | 697 GB |
| 32 | 658 GB | 774 GB |
| 64 | 697 GB | 929 GB |
On one card, with vLLM's small-card settings (2,048 tokens a step). Cards of 70 GB and more reserve more per request and more overhead — each row above uses its own card's settings.
Estimates, not measurements, checked against published vLLM startup logs. The weights are the build's file size; the cache is calculated from its config (grouped-query attention); the overhead is an estimate of vLLM's own memory with that card's default settings. "Requests at once" is how many requests of that length vLLM admits — its reservation at full length, with --max-model-len set to that length; requests that stay shorter fit more. "Longest single request" is the most one request can hold there: below the model's maximum, vLLM starts only with --max-model-len set at or under it. "Counted memory" is vLLM's default 92 % of what CUDA reports for the card (the DGX Spark: about 100 GiB of its shared 128 GB). A tensor-parallel split pools the cards' memory and speeds each token up, at the cost of the link between them; a layer split (llama.cpp) holds more but does not make one request faster. Assumes vLLM 0.10 or later.
From the model card
[🏠 Homepage][website] · [📰 Technical Blog][blog] · [💻 GitHub][github]
Naive-N0.5-Flash is an open-weight 309B MoE model with 15.5B active parameters, built for coding and AI R&D. It supports a native 1M-token context window through a hybrid of Sliding-Window Attention (SWA) and lightweight DeepSeek Sparse Attention (DSA), with no full-attention layers.
| Property | Specification |
|---|---|
| Architecture | Mixture-of-Experts (MoE) |
| Total parameters | 309B |
| Active parameters | 15.5B |
| Context length | Native 1M tokens |
| Transformer layers | 48 |
| Attention-layer composition | 39 SWA layers + 9 DSA layers |
| Attention mechanism | Hybrid SWA–DSA |
| SWA window | 128 tokens |
| DSA token selection | Top 2,048 tokens for backbone attention |
| DSA KV groups | 4 (GQA4) |
| Indexer query heads | 16 |
Naive-N0.5-Flash builds on the open-weight MiMo-V2.5 base model, which has a simple architecture with strong foundational capabilities in world knowledge and deep research. Most layers use Sliding-Window Attention (SWA), whose per-token decoding cost does not grow with context length, while a small number of global-attention layers preserve long-range information. At million-token context lengths, however, these global-attention layers account for much of the decoding overhead.
Naive-N0.5-Flash replaces the global-attention layers with DeepSeek Sparse Attention (DSA). A lightweight indexer scores the full history, while the backbone computes attention only over a selected subset of tokens. Although the indexer still scans the full history and the full KV cache is retained, sparse attention substantially reduces attention computation and memory access. Adapting the model to this new attention structure was one objective of continued pretraining.
The network consists of eight six-layer modules. A standard module contains five SWA layers followed by one DSA layer, with the first layer of the first module also replaced by DSA. SWA uses a 128-token window, while DSA selects the top 2,048 tokens for backbone attention. Both attention types incorporate sink bias.
Unlike the original MLA-based DSA implementation, Naive-N0.5-Flash replaces MLA with grouped-query attention (GQA) using four KV groups. For the architecture design process and indexer efficiency comparison, see model architecture in the [technical blog][blog].
Following the architectural changes, Naive-N0.5-Flash completed 3.25T tokens of multi-stage training with a native 1M-token context window: 50B tokens of Indexer Warmup, 3T tokens of Sparse Attention Training, and 200B tokens of Learning Rate Decay. This process adapted the model to its new sparse attention architecture while substantially improving its AI R&D and coding capabilities. See the [technical blog][blog] for training details.
[![Coding benchmarks comparing Naive-N0.5-Flash with other models across seven software engineering and agentic tasks.][coding-figure]][coding-pdf]
[![AI R&D benchmarks covering PostTrainBench, MLE-bench-30, PaperBench, SOL-ExecBench, NanoChat AutoResearch, and NanoGPT SpeedRun.][ai-rd-figure]][ai-rd-pdf]
Evaluation setup. Unless otherwise noted, our evaluations of Naive-N0.5-Flash use Claude Code 2.1.207 with a 1M-token context window, temperature 1.0, and top-p 0.95. The harness exposes only basic file I/O and Bash tools.
Sources for reported benchmark scores are as follows:
Quoted from the model card on Hugging Face — the full card is behind the Hugging Face link above.