Model reference · open weights
BananaMind-2-Pro is an open-weight language model from BananaMind. AxForge deploys and operates it for you on dedicated EU-owned hardware — with the licence handled where one is required.
Available as managed deployment — configured and operated for you on dedicated EU hardware, quoted per deployment.
What it is
| Released by | BananaMind |
|---|---|
| Type | Language models |
| Task | Text gen |
| Parameters (lead) | 160M |
| Context | 3k tokens |
| Runs with | transformers |
| Released | 2026-08-14 |
| Popularity | 1k downloads / month |
| Licence | Commercial licence needed |
About
BananaMind-2-Pro is a 138,971,520-parameter decoder-only base language model trained from scratch by BananaMind. The completed pretraining run processed 99,999,449,088 tokens—the scheduled realization of a 100B-token curriculum—over 184,954 optimizer steps.
The model has a 3,072-token context window and a custom 32,768-token digit-aware byte-level BPE tokenizer. Its architecture uses grouped-query attention, QK normalization, RoPE, SwiGLU, RMSNorm, tied input/output embeddings, and a KV cache for generation.
This is a base model, not an instruction-tuned or chat model. Use continuation-style prompts and load the repository with trust_remote_code=True.
| Field | Value |
|---|---|
| Release type | Final base checkpoint |
| Checkpoint step | 184,953 |
| Optimizer steps completed | 184,954 |
| Tokens seen | 99,999,449,088 |
| Training target | 100B tokens |
| Final phase | Quality finish |
| Training status | Complete |
| Evaluation status | Complete |
| Field | Value |
|---|---|
| Parameters | 138,971,520 |
| Architecture | BananaMind2Pro decoder-only Transformer |
| Layers | 24 |
| Hidden size | 640 |
| Intermediate size | 1,920 |
| Attention heads | 8 |
| KV heads | 4 |
| Head dimension | 80 |
| Attention style | Grouped-query attention with QK norm |
| MLP | SwiGLU |
| Position embeddings | RoPE |
| RoPE theta | 100,000 |
| Normalization | RMSNorm |
| RMSNorm epsilon | 1e-6 |
| Vocabulary size | 32,768 |
| Context length | 3,072 |
| Embeddings | Tied input/output embeddings |
| Generation cache | KV cache supported |
| Weight format | safetensors |
| HF architecture | BananaMind2ProForCausalLM |
| HF model type | bananamind2_pro |
The model maps tokens through a tied input embedding, 24 pre-normalized Transformer blocks, a final RMSNorm, and the tied language-model head. Each block contains causal grouped-query self-attention followed by a SwiGLU feed-forward network, with residual connections around both sublayers.
Eight query heads share four key/value heads, reducing KV-cache size while retaining multi-head query capacity. QK normalization stabilizes attention logits, and RoPE supplies position information without learned absolute-position embeddings. During autoregressive generation, each layer can reuse cached keys and values instead of recomputing the full prefix.
These results describe the exact final checkpoint at step 184,953. ARC Easy, ARC Challenge, PIQA, and HellaSwag use zero-shot acc_norm,none. ArithMark 3 uses length-normalized continuation accuracy, while ArithMark 2 uses raw continuation accuracy. Code Elo is the Base Bench 1.1 code-completion category; Base Bench 1.1 Elo is the complete 350-item suite.
| Benchmark | BananaMind-2-Pro | BananaMind-2-Pro-Preview | GPT-X2.5-135M | BananaMind-2-Medium | GPT-2 |
|---|---|---|---|---|---|
| Trainable parameters | 139M | 139M | 135M | 49.6M | 124M |
| ARC Easy | 53.58% | 51.01% | 51.81% | 43.81% | 39.35% |
| ARC Challenge | 27.82% | 27.13% | 29.18% | 25.34% | 22.35% |
| PIQA | 67.52% | 66.76% | 69.42% | 61.86% | 62.08% |
| HellaSwag | 42.78% | 39.83% | 40.57% | 32.43% | 31.26% |
| ArithMark 3 | 38.20% | 38.90% | 38.10% | 36.20% | 35.70% |
| ArithMark 2 | 32.08% | 28.60% | N/A | 28.20% | 26.48% |
| INT Index | 24.96 | 23.04 | 25.17 | 15.37 | N/A |
| Code Elo | 1407 | 1295 | 1253 | 1034 | 996 |
| Base Bench 1.1 Elo | 1124 | 1106 | 1106 | 1034 | 996 |
The INT Index chance-normalizes HellaSwag, the mean of ARC Easy and ARC Challenge, PIQA, and ArithMark 3. ArithMark 2 is not included:
N(s, c) = 100 * (s - c) / (100 - c)
INT = [N(H,25) + N((ARC_E+ARC_C)/2,25) + N(P,50) + 0.65*N(A3,25)] / 3.65
Training compute is estimated consistently as 6 × parameters × training tokens. Using the exact 138,971,520-parameter count gives 83,382.91 PFLOPs for Pro Final at 100B tokens and 43,279.49 PFLOPs for Pro Preview at 51.904512B tokens. GPT-X2.5 uses 135M parameters, 75B tokens, and a 2,048-token context window; context length is not a separate term in this 6ND approximation.
| Model | Estimated training compute | INT Index |
|---|---|---|
| BananaMind-2-Pro Final | 83,382.91 PFLOPs | 24.96 |
| BananaMind-2-Pro Preview | 43,279.49 PFLOPs | 23.04 |
| GPT-X2.5-135M | 60,750.00 PFLOPs | 25.17 |
| GPT-X2-125M | 56,286.75 PFLOPs | 23.36 |
| GPT-X-125M | 11,210.56 PFLOPs | 19.94 |
| Supra2-100M | 18,000.00 PFLOPs | 19.41 |
| SmolLM-135M | 484,254.03 PFLOPs | 25.74 |
| BananaMind-2-Medium | 14,867.33 PFLOPs | 15.37 |
| OPT-125M | 135,000.00 PFLOPs | 13.80 |
This progression contains 41 complete 350-item evaluations from 2.70B tokens through the final 99.999B-token checkpoint. Every point uses CUDA, bfloat16, and batch size 1 for consistency. The final batch-1 endpoint is 1132 Elo, 236/350 correct (67.43%), and 64.69% weighted accuracy. The primary comparison table above retains the separately measured batch-32 result.
The final model passed 233 of 350 cases for 66.57% raw accuracy and 63.73% weighted accuracy. Its overall fixed-item rating is 1124 Elo.
| Category | Elo | Correct | Accuracy | Weighted accuracy |
|---|---|---|---|---|
| Language completion | 1570 | 50/50 | 100.00% | 100.00% |
| Commonsense | 1185 | 40/50 | 80.00% | 78.86% |
| World knowledge | 1168 | 40/50 | 80.00% | 77.12% |
| Context tracking | 938 | 22/50 | 44.00% | 41.72% |
| Quantitative | 886 | 15/50 | 30.00% | 28.71% |
| Logical reasoning | 1059 | 24/50 | 48.00% | 44.51% |
| Code completion | 1407 | 42/50 | 84.00% | 85.67% |
| Overall | 1124 | 233/350 | 66.57% | 63.73% |
Base Bench was evaluated locally with CUDA, bfloat16, and batch size 32. ArithMark 3 used CUDA bfloat16 with batch size 64; ArithMark 2 used CUDA bfloat16 with batch size 1. Resul
From the published model card. Full card on the HuggingFace links in the sidebar.
Using it via the API
Once AxForge deploys bananamind-2-pro for you, it answers on the OpenAI-compatible API — the same base URL and keys as every other model. (bananamind-2-pro below is illustrative; you get the exact model name on deployment.)
$ curl -sS https://api.axforge.ai/v1/chat/completions \
-H "Authorization: Bearer $AXFORGE_API_KEY" \
-H "Content-Type: application/json" \
-d '{"model":"bananamind-2-pro","messages":[{"role":"user","content":"Hello"}]}'
Create an account — your API key is available in the console. 3M free tokens every 30 days with every new account.