Model reference · open weights
SparkAI-llama is an open-weight language model from vedantjadhav701. AxForge deploys and operates it for you on dedicated EU-owned hardware — with the licence handled where one is required.
Available as managed deployment — configured and operated for you on dedicated EU hardware, quoted per deployment.
What it is
| Released by | vedantjadhav701 |
|---|---|
| Type | Language models |
| Task | Text gen |
| Parameters (lead) | 48M |
| Context | 1k tokens |
| Runs with | transformers |
| Based on | vedantjadhav701/SparkAI-47m-llama-10b-token |
| Released | 2026-08-27 |
| Popularity | 653 downloads / month |
| Licence | Open weights |
About
Instruction-tuned checkpoint of SparkAI-47M-Llama (~48M parameter decoder-only transformer), fine-tuned for chat and instruction following.
🌟 Key Highlights & Unique Features
- ⚡ Ultra-Low Memory Footprint (~95.4 MB): Fits in under 100MB of RAM, making it suitable for edge devices, mobile apps, WebGPU, and microcontrollers.
- 🏋️ Data-Saturated Pretraining (10 Billion Tokens): Pretrained on 10B tokens (210 tokens/param) of high-quality FineWeb-Edu + Cosmopedia-v2 text, providing an empirical benchmark on capacity saturation for sub-50M models.
- 🏗️ Modern LLaMA 3 Architecture: Built with Grouped Query Attention (GQA), SwiGLU activations, RoPE positional encodings, RMSNorm pre-normalization, and tied embeddings.
- 💬 Full ChatML SFT Alignment: Fine-tuned with ChatML `` instruction formatting and template support (
chat_template.jinja).
vedantjadhav701/SparkAI-47m-llama-10b-tokensample-100BT) + Cosmopedia-v2 (85% / 15% mix, 10.00B tokens)| Tokens | Perplexity |
|---|---|
| 630M | 43.49 |
| 3.77B | 31.30 |
| 7.00B | — |
| 10.00B | 31.46 |
Note on Saturation: Perplexity plateaued between 3.77B and 10.00B tokens despite continued training, indicating the model has saturated its representational capacity at this size.
| Feature | Typical Sub-50M Models | SparkAI-47M-Llama / Instruct |
|---|---|---|
| Token Budget | ~1B – 2B tokens | 10.00 Billion Tokens (210 tokens/param) |
| Data Quality | Raw web text / C4 | FineWeb-Edu (85%) + Cosmopedia-v2 (15%) |
| Architecture | Basic MHA / GPT-2 style | Modern LLaMA 3 (GQA, SwiGLU, RoPE, RMSNorm) |
| Model Size | ~100MB – 200MB | ~95.4 MB (model.safetensors) |
| SFT Alignment | Rare / None | Instruction-tuned with ChatML (chat_template.jinja) |
| Benchmarking | Few metrics | Empirical capacity saturation documented at 10B tokens |
transformersimport torch
from transformers import AutoModelForCausalLM, AutoTokenizer
repo_id = "vedantjadhav701/SparkAI-47m-llama-instruct"
tokenizer = AutoTokenizer.from_pretrained(repo_id)
model = AutoModelForCausalLM.from_pretrained(repo_id)
messages = [
{"role": "user", "content": "What is a computer program?"}
]
prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=80, do_sample=True, temperature=0.6)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
pip install -r requirements.txt
python app.py
Open http://127.0.0.1:7860 in your web browser.
From the published model card. Full card on the HuggingFace links in the sidebar.
Using it via the API
Once AxForge deploys sparkai-llama for you, it answers on the OpenAI-compatible API — the same base URL and keys as every other model. (sparkai-llama below is illustrative; you get the exact model name on deployment.)
$ curl -sS https://api.axforge.ai/v1/chat/completions \
-H "Authorization: Bearer $AXFORGE_API_KEY" \
-H "Content-Type: application/json" \
-d '{"model":"sparkai-llama","messages":[{"role":"user","content":"Hello"}]}'
Create an account — your API key is available in the console. 3M free tokens every 30 days with every new account.