Model reference · open weights
stablelm-2-zephyr-1 is an open-weight language model from stabilityai. AxForge deploys and operates it for you on dedicated EU-owned hardware — with the licence handled where one is required.
Available as managed deployment — configured and operated for you on dedicated EU hardware, quoted per deployment.
What it is
| Maker | stabilityai |
|---|---|
| Type | Language models |
| Task | Text gen |
| Parameters (lead) | 1.6B |
| Runs with | transformers |
| Released | 2024-01-19 |
| Popularity | 11k downloads / month |
| Licence | Commercial licence needed |
About
Stable LM 2 Zephyr 1.6B is a 1.6 billion parameter instruction tuned language model inspired by HugginFaceH4's Zephyr 7B training pipeline. The model is trained on a mix of publicly available datasets and synthetic datasets, utilizing Direct Preference Optimization (DPO).
StableLM 2 Zephyr 1.6B uses the following instruction format:
Which famous math number begins with 1.6 ...?
The number you are referring to is 1.618033988749895. This is the famous value known as the golden ratio
This format is also available through the tokenizer's apply_chat_template method:
from transformers import AutoModelForCausalLM, AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained('stabilityai/stablelm-2-zephyr-1_6b')
model = AutoModelForCausalLM.from_pretrained(
'stabilityai/stablelm-2-zephyr-1_6b',
device_map="auto"
)
prompt = [{'role': 'user', 'content': 'Which famous math number begins with 1.6 ...?'}]
inputs = tokenizer.apply_chat_template(
prompt,
add_generation_prompt=True,
return_tensors='pt'
)
tokens = model.generate(
inputs.to(model.device),
max_new_tokens=1024,
temperature=0.5,
do_sample=True
)
print(tokenizer.decode(tokens[0], skip_special_tokens=False))
StableLM 2 Zephyr 1.6B model is an auto-regressive language model based on the transformer decoder architecture.lm@stability.aiThe dataset is comprised of a mixture of open datasets large-scale datasets available on the HuggingFace Hub:
| Model | Size | MT-Bench |
|---|---|---|
| Mistral-7B-Instruct-v0.2 | 7B | 7.61 |
| Llama2-Chat | 70B | 6.86 |
| stablelm-zephyr-3b | 3B | 6.64 |
| MPT-30B-Chat | 30B | 6.39 |
| stablelm-2-zephyr-1.6b | 1.6B | 5.42 |
| Falcon-40B-Instruct | 40B | 5.17 |
| Qwen-1.8B-Chat | 1.8B | 4.95 |
| dolphin-2.6-phi-2 | 2.7B | 4.93 |
| phi-2 | 2.7B | 4.29 |
| TinyLlama-1.1B-Chat-v1.0 | 1.1B | 3.46 |
| Model | Size | Average | ARC Challenge (acc_norm) | HellaSwag (acc_norm) | MMLU (acc_norm) | TruthfulQA (mc2) | Winogrande (acc) | Gsm8k (acc) |
|---|---|---|---|---|---|---|---|---|
| microsoft/phi-2 | 2.7B | 61.32% | 61.09% | 75.11% | 58.11% | 44.47% | 74.35% | 54.81% |
| stabilityai/stablelm-2-zephyr-1_6b | 1.6B | 49.89% | 43.69% | 69.34% | 41.85% | 45.21% | 64.09% | 35.18% |
| microsoft/phi-1_5 | 1.3B | 47.69% | 52.90% | 63.79% | 43.89% | 40.89% | 72.22% | 12.43% |
| stabilityai/stablelm-2-1_6b | 1.6B | 45.54% | 43.43% | 70.49% | 38.93% | 36.65% | 65.90% | 17.82% |
| mosaicml/mpt-7b | 7B | 44.28% | 47.70% | 77.57% | 30.80% | 33.40% | 72.14% | 4.02% |
| KnutJaegersberg/Qwen-1_8B-Llamaified* | 1.8B | 44.75% | 37.71% | 58.87% | 46.37% | 39.41% | 61.72% | 24.41% |
| openlm-research/open_llama_3b_v2 | 3B | 40.28% | 40.27% | 71.60% | 27.12% | 34.78% | 67.01% | 0.91% |
| iiuae/falcon-rw-1b | 1B | 37.07% | 35.07% | 63.56% | 25.28% | 35.96% | 62.04% | 0.53% |
| TinyLlama/TinyLlama-1.1B-3T | 1.1B | 36.40% | 33.79% | 60.31% | 26.04% | 37.32% | 59.51% | 1.44% |
StableLM 2 Zephyr 1.6B was trained on the Stability AI cluster across 8 nodes with 8 A100 80GBs GPUs for each nodes.The model is intended to be used in chat-like applications. Developers must evaluate the model for safety performance in their specific use case. Read more about safety and limitations below.
From the published model card. Full card on the HuggingFace links in the sidebar.
Using it via the API
Once AxForge deploys stablelm-2-zephyr-1 for you, it answers on the OpenAI-compatible API — the same base URL and keys as every other model. (stablelm-2-zephyr-1 below is illustrative; you get the exact model name on deployment.)
$ curl -sS https://api.axforge.ai/v1/chat/completions \
-H "Authorization: Bearer $AXFORGE_API_KEY" \
-H "Content-Type: application/json" \
-d '{"model":"stablelm-2-zephyr-1","messages":[{"role":"user","content":"Hello"}]}'
Create an account — your API key is available in the console. 5M tokens/month currently included with every new account at launch.