Model reference · open weights
voilum-1 is an open-weight language model from elisepaul. AxForge deploys and operates it for you on dedicated EU-owned hardware — with the licence handled where one is required.
Available as managed deployment — configured and operated for you on dedicated EU hardware, quoted per deployment.
What it is
| Released by | elisepaul |
|---|---|
| Type | Language models |
| Task | Text gen |
| Parameters (lead) | 464M |
| Runs with | pytorch |
| Released | 2026-09-12 |
| Popularity | 536 downloads / month |
| Licence | Open weights |
About
We introduce Voilum-1, a high-efficiency 463.6-million parameter (100.8M active parameters per token) sparse Mixture-of-Experts (MoE) Small Language Model (SLM) specialized for deterministic, mathematically rigorous Python code synthesis, numerical scientific computing, and deep learning systems implementation. Trained across 12.0 billion tokens of curated algorithmic and mathematical data, aligned via ChatML instruction tuning, and reinforced through a 6-tier curriculum of execution-grounded Group Relative Policy Optimization (GRPO), Voilum-1 solves complex machine learning tasks from conceptual specifications without code spoon-feeding. In empirical sliding-window evaluations, Voilum-1 achieves a 99.3% greedy pass rate on challenging applied machine learning and numerical systems benchmarks while operating within an ultra-compact memory footprint of 884 MB in BF16 SafeTensors.
Voilum-1 employs a decoder-only sparse Mixture-of-Experts architecture incorporating Grouped-Query Attention (GQA), Rotary Position Embeddings (RoPE), RMSNorm, and SwiGLU gating:
| Architectural Hyperparameter | Specification | Description |
|---|---|---|
| Total Parameters | 463,553,024 (463.6M) | Full weights memory footprint (884 MB in BF16) |
| Active Parameters / Token | 100,800,000 (~100.8M) | Parameters engaged in each forward pass |
| Transformer Layers ( (N_{\text{layers}}) ) | 16 | Pre-norm decoder blocks |
| Hidden Dimension ( (d_{\text{model}}) ) | 512 | Model channel representation dimension |
| Feed-Forward Routing | 8 Experts (Top-1 / Top-2) | Sparse token-level gating with softmax load balancing |
| Expert Intermediate Dim | 2,048 | SwiGLU projection dimension per expert |
| Attention Mechanism | Grouped-Query Attention (GQA) | 8 Query heads, 2 Key-Value heads (4:1 compression) |
| Head Dimension ( (d_k) ) | 64 | Per-head projection dimension |
| Positional Encoding | Rotary Embeddings (RoPE) | Base frequency ( \theta = 10,000.0 ) |
| Normalization | RMSNorm | Zero-mean centering omitted for scale invariance: ( \epsilon = 10^{-6} ) |
| Activation Function | SwiGLU | Gated Swish-linear non-linearity |
| Context Length ( (L_{\text{ctx}}) ) | 2,048 tokens | Native autoregressive training horizon |
| Vocabulary Size ( (V) ) | 49,152 | Byte-Pair Encoding (StarCoder2 BPE) |
Each decoder layer executes the following standardized pre-norm transformation sequence:
$$ \text{RMSNorm}(x, \epsilon=10^{-6}) = \frac{x}{\sqrt{\frac{1}{d}\sum_{i=1}^{d} x_i^2 + \epsilon}} \odot \gamma $$
$$ \text{Attention}(Q, K, V) = \text{softmax}\left(\frac{Q K^T}{\sqrt{d_k}}\right) V $$
Pre-FFN Normalization: The attention output is normalized through a second RMSNorm layer.
Sparse Mixture-of-Experts Feed-Forward: A learnable routing gate computes softmax probabilities over 8 experts. The top-2 activated experts evaluate independent SwiGLU transformations with intermediate projection dimension 2,048:
$$ \text{SwiGLU}(x) = \left(\text{Swish}(x W_{\text{gate}}) \otimes (x W_{\text{up}})\right) W_{\text{down}} $$
The routed expert outputs are dynamically weighted by routing probabilities, summed, and added to the residual stream.
Voilum-1 was developed through a three-stage progressive optimization pipeline:
Stage 1: Base Pretraining (12.0 Billion Non-Repeating Tokens) Pretrained on a highly curated corpus spanning scientific Python (NumPy, SciPy, Pandas), PyTorch deep learning internals, algorithms, and numerical mathematics. Optimized with AdamW (peak learning rate ( 5 \times 10^{-4} ), linear warmup, cosine decay, bfloat16 mixed precision).
Stage 2: Direct Instruction SFT Alignment
Aligned using ChatML formatting (/) under a strict Zero-Thinking Token policy. The model is trained to produce clean, direct, executable code without conversational boilerplate or unstructured internal monologue.
Stage 3: Pedagogical T-GRPO Reinforcement Learning (Tiers 1 to 6) Post-trained using Group Relative Policy Optimization (GRPO) with signed advantage baselines (( G = 4 ) rollouts per prompt) and KL penalty against the Stage 2 reference policy. Evaluated via an automated, isolated compiler and unit-test execution sandbox across 6 pedagogical mastery tiers.
Unlike conventional coding benchmarks that spoon-feed mathematical equations directly inside the prompt (e.g. rms = torch.sqrt(torch.mean(x**2) + eps)), Voilum-1 is reinforced against authentic engineering and applied system specifications:
All tiers were evaluated under strict deterministic greedy decoding (( T = 0.0, \text{argmax} )) over rolling evaluation windows of ( W = 15 ) (300 training steps):
$$ \text{Rolling Greedy Pass Rate} = \frac{1}{W} \sum_{k=1}^{W} \text{GreedyEval}_k \ge 25.0% \implies \text{Tier Mastered} $$
| Curriculum
From the published model card. Full card on the HuggingFace links in the sidebar.
Using it via the API
Once AxForge deploys voilum-1 for you, it answers on the OpenAI-compatible API — the same base URL and keys as every other model. (voilum-1 below is illustrative; you get the exact model name on deployment.)
$ curl -sS https://api.axforge.ai/v1/chat/completions \
-H "Authorization: Bearer $AXFORGE_API_KEY" \
-H "Content-Type: application/json" \
-d '{"model":"voilum-1","messages":[{"role":"user","content":"Hello"}]}'
Create an account — your API key is available in the console. 3M free tokens every 30 days with every new account.