Model reference · open weights
clio-legacy is an open-weight language model from NovelAI. clio-v1-legacy (BF16) weighs 2.0 GB; the smallest configuration that runs it is RTX 3060 12 GB.
What it is
| Released by | NovelAI |
|---|---|
| Type | Language models |
| Task | Text gen |
| Parameters (lead) | 3.0B |
| Context | 8,192 tokens |
| Runs with | transformers |
| Released | 2026-09-24 |
| Popularity | 2k downloads / month |
| Weights | 2.0 GB (clio-v1-legacy (BF16), file size) |
| Licence | Open, with conditions |
What it runs on
Weights 2.0 GB (file size) · KV cache 315 MB per 1,000 tokens of context, at 16 bits (vLLM's default for this build; an 8-bit cache halves it) · runtime overhead from 495 MB on a small card · context up to 8,192 tokens.
| Card | Requests at once 8K tokens each | Requests at once 32K tokens each | Longest single request | Counted memory |
|---|---|---|---|---|
| RTX 3060 12 GB | 3 | — | all 8K | 11.6 GB |
| RTX 4060 Ti 16 GB | 5 | — | all 8K | 15.4 GB |
| RTX 3090 24 GB | 8 | — | all 8K | 23.4 GB |
| RTX 4090 24 GB | 8 | — | all 8K | 23.4 GB |
| RTX 5090 32 GB | 11 | — | all 8K | 31.0 GB |
| L40S 48 GB | 16 | — | all 8K | 44.0 GB |
| A100 80 GB | 29 | — | all 8K | 78.2 GB |
| H100 80 GB | 29 | — | all 8K | 78.1 GB |
| RTX PRO 6000 Blackwell 96 GB | 35 | — | all 8K | 93.8 GB |
| DGX Spark (GB10) 128 GB unified | 40 | — | all 8K | 107 GB |
| H200 141 GB | 52 | — | all 8K | 138 GB |
| B200 180 GB | 67 | — | all 8K | 176 GB |
| 2× RTX 3060 12 GB split by layers (llama.cpp) | 7 | — | all 8K | 11.6 GB a card |
| 2× RTX 4060 Ti 16 GB split by layers (llama.cpp) | 10 | — | all 8K | 15.4 GB a card |
| Requests at once | 8K tokens each | 32K tokens each |
|---|---|---|
| 1 | 5.1 GB | — |
| 5 | 15.4 GB | — |
| 8 | 23.1 GB | — |
| 16 | 43.8 GB | — |
| 32 | 85.1 GB | — |
| 64 | 168 GB | — |
On one card, with vLLM's small-card settings (2,048 tokens a step). Cards of 70 GB and more reserve more per request and more overhead — each row above uses its own card's settings.
Estimates, not measurements, checked against published vLLM startup logs. The weights are the build's file size; the cache is calculated from its config (multi-head attention); the overhead is an estimate of llama.cpp's own memory with that card's default settings. "Requests at once" is how many requests of that length vLLM admits — its reservation at full length, with --max-model-len set to that length; requests that stay shorter fit more. "Longest single request" is the most one request can hold there: below the model's maximum, vLLM starts only with --max-model-len set at or under it. "Counted memory" is vLLM's default 92 % of what CUDA reports for the card (the DGX Spark: about 100 GiB of its shared 128 GB). A tensor-parallel split pools the cards' memory and speeds each token up, at the cost of the link between them; a layer split (llama.cpp) holds more but does not make one request faster. Assumes vLLM 0.10 or later.
From the model card
Clio is the first fully original large language model that we pretrained from scratch on our Shoggy H100 cluster. At 3 billion paramters, she runs very fast, but thanks to the, for her time, very large training volume, her writing outperforms much bigger models like Euterpe and Krake. She was trained on our custom Nerdstash dataset, with our custom Nerdstash Tokenizer V1.
Back when she was released, Clio was very competitive across various evaluation metrics, even when compared with much bigger models. The following table shows the performance of the base model before being finetuned for story telling:
Nowadays, much stronger models, like Kayra, Erato and Xialong are available on NovelAI, so before long, Clio will get to enjoy a peaceful retirement.
So, in anticipation of this, for the sake of nostalgia, posterity, and historic preservation, we are releasing the weights of our Clio model and her modules publicly on Huggingface Hub under the GPL-2.0 (not "or later") license.
We managed to find an existing model class inside Huggingface Transformers which fits the shape of our architecture for Clio, at least once an additional config flag is set, so no custom code is needed to run Clio. Model files in GGUF format are provided as well.
You can find your very own Clio right here where you are!
Quoted from the model card on Hugging Face — the full card is behind the Hugging Face link above.