Model reference · open weights
NanoDex is an open-weight language model from DedeProGames. AxForge deploys and operates it for you on dedicated EU-owned hardware — with the licence handled where one is required.
Available as managed deployment — configured and operated for you on dedicated EU hardware, quoted per deployment.
What it is
| Released by | DedeProGames |
|---|---|
| Type | Language models |
| Task | Text gen |
| Parameters (lead) | 1M |
| Context | 512 tokens |
| Runs with | transformers |
| Released | 2026-09-12 |
| Popularity | 532 downloads / month |
| Licence | Commercial licence needed |
About
A 1,062,272-parameter decoder-only language model pre-trained from scratch on fineweb-edu, using the NanoDex Trainer Space.
A standard LlamaForCausalLM decoder-only transformer — SiLU MLP, RMSNorm,
rotary position embeddings, grouped-query attention, tied embeddings, no biases —
scaled down in width and depth to fit the parameter budget.
| Parameters | 1,062,272 |
| Hidden size | 128 |
| Layers | 5 |
| Attention heads | 8 (KV: 4) |
| FFN size | 288 |
| Context length | 512 |
| Vocab | 2,048 (custom BPE trained on fineweb-edu) |
| Tokens seen | 999,817,216 |
| Steps | 3,814 |
| Tokens / step | 262,144 |
| Optimizer | AdamW(0.9, 0.95) wd=0.1 clip=1.0 |
| LR schedule | warmup 2% + cosine to 10% (peak 3e-03) |
| Final loss | 3.3092 (ppl 27.4) |
| Wall time | 43.8 min |
| Trained by | @DedeProGames |
from transformers import AutoModelForCausalLM, AutoTokenizer
tok = AutoTokenizer.from_pretrained("DedeProGames/NanoDex-1M")
model = AutoModelForCausalLM.from_pretrained("DedeProGames/NanoDex-1M")
ids = tok("The mitochondria is", return_tensors="pt").input_ids
print(tok.decode(model.generate(ids, max_new_tokens=60, do_sample=True,
temperature=0.8, top_k=50)[0]))
This is a nano-scale research artifact. At this parameter count and token budget the model learns word shapes, common collocations and a little syntax — it is not a useful assistant and its output is not factual. It exists to make "pre-train a transformer from scratch" something you can actually watch happen.
From the published model card. Full card on the HuggingFace links in the sidebar.
Using it via the API
Once AxForge deploys nanodex for you, it answers on the OpenAI-compatible API — the same base URL and keys as every other model. (nanodex below is illustrative; you get the exact model name on deployment.)
$ curl -sS https://api.axforge.ai/v1/chat/completions \
-H "Authorization: Bearer $AXFORGE_API_KEY" \
-H "Content-Type: application/json" \
-d '{"model":"nanodex","messages":[{"role":"user","content":"Hello"}]}'
Create an account — your API key is available in the console. 3M free tokens every 30 days with every new account.