Model reference · open weights
WizardLM-2-8x is an open-weight language model from alpindale. AxForge deploys and operates it for you on dedicated EU-owned hardware — with the licence handled where one is required.
Available as managed deployment — configured and operated for you on dedicated EU hardware, quoted per deployment.
What it is
| Released by | alpindale |
|---|---|
| Type | Language models |
| Task | Text gen |
| Parameters (lead) | 140.6B |
| Context | 64k tokens |
| Runs with | transformers |
| Released | 2024-04-16 |
| Popularity | 9k downloads / month |
| Licence | Open weights |
About
🏠 WizardLM-2 Release Blog 🤗 HF Repo •🐱 Github Repo • 🐦 Twitter • 📃 [WizardLM] • 📃 [WizardCoder] • 📃 [WizardMath]
We introduce and opensource WizardLM-2, our next generation state-of-the-art large language models, which have improved performance on complex chat, multilingual, reasoning and agent. New family includes three cutting-edge models: WizardLM-2 8x22B, WizardLM-2 70B, and WizardLM-2 7B.
For more details of WizardLM-2 please read our release blog post and upcoming paper.
MT-Bench
We also adopt the automatic MT-Bench evaluation framework based on GPT-4 proposed by lmsys to assess the performance of models. The WizardLM-2 8x22B even demonstrates highly competitive performance compared to the most advanced proprietary models. Meanwhile, WizardLM-2 7B and WizardLM-2 70B are all the top-performing models among the other leading baselines at 7B to 70B model scales.
Human Preferences Evaluation
We carefully collected a complex and challenging set consisting of real-world instructions, which includes main requirements of humanity, such as writing, coding, math, reasoning, agent, and multilingual. We report the win:loss rate without tie:
We built a fully AI powered synthetic training system to train WizardLM-2 models, please refer to our blog for more details of this system.
❗Note for model system prompts usage:
A chat between a curious user and an artificial intelligence assistant. The assistant gives helpful,
detailed, and polite answers to the user's questions. USER: Hi ASSISTANT: Hello.
USER: Who are you? ASSISTANT: I am WizardLM.......
We provide a WizardLM-2 inference demo code on our github.
Detailed results can be found here
| Metric | Value |
|---|---|
| Avg. | 32.61 |
| IFEval (0-Shot) | 52.72 |
| BBH (3-Shot) | 48.58 |
| MATH Lvl 5 (4-Shot) | 22.28 |
| GPQA (0-shot) | 17.56 |
| MuSR (0-shot) | 14.54 |
| MMLU-PRO (5-shot) | 39.96 |
From the published model card. Full card on the HuggingFace links in the sidebar.
Benchmarks
As published on the model card — the maker's own numbers, not measured by AxForge.
| Task | Dataset | Metric | Score |
|---|---|---|---|
| Text Generation | IFEval (0-Shot) | strict accuracy | 52.720 |
| Text Generation | BBH (3-Shot) | normalized accuracy | 48.580 |
| Text Generation | MATH Lvl 5 (4-Shot) | exact match | 22.280 |
| Text Generation | GPQA (0-shot) | acc_norm | 17.560 |
| Text Generation | MuSR (0-shot) | acc_norm | 14.540 |
| Text Generation | MMLU-PRO (5-shot) | accuracy | 39.960 |
Using it via the API
Once AxForge deploys wizardlm-2-8x for you, it answers on the OpenAI-compatible API — the same base URL and keys as every other model. (wizardlm-2-8x below is illustrative; you get the exact model name on deployment.)
$ curl -sS https://api.axforge.ai/v1/chat/completions \
-H "Authorization: Bearer $AXFORGE_API_KEY" \
-H "Content-Type: application/json" \
-d '{"model":"wizardlm-2-8x","messages":[{"role":"user","content":"Hello"}]}'
Create an account — your API key is available in the console. 3M free tokens every 30 days with every new account.