Model reference · open weights
Qwen1.5-MoE-A is an open-weight language model from Qwen. AxForge deploys and operates it for you on dedicated EU-owned hardware — with the licence handled where one is required.
Available as managed deployment — configured and operated for you on dedicated EU hardware, quoted per deployment.
What it is
| Released by | Qwen |
|---|---|
| Type | Language models |
| Task | Text gen · MoE |
| Parameters (lead) | 14.3B |
| Context | 8k tokens |
| Runs with | transformers |
| Released | 2024-02-29 |
| Popularity | 716k downloads / month |
| Licence | Commercial licence needed |
About
Qwen1.5-MoE is a transformer-based MoE decoder-only language model pretrained on a large amount of data.
For more details, please refer to our blog post and GitHub repo.
Qwen1.5-MoE employs Mixture of Experts (MoE) architecture, where the models are upcycled from dense language models. For instance, Qwen1.5-MoE-A2.7B is upcycled from Qwen-1.8B. It has 14.3B parameters in total and 2.7B activated parameters during runtime, while achieving comparable performance to Qwen1.5-7B, it only requires 25% of the training resources. We also observed that the inference speed is 1.74 times that of Qwen1.5-7B.
The code of Qwen1.5-MoE has been in the latest Hugging face transformers and we advise you to build from source with command pip install git+https://github.com/huggingface/transformers, or you might encounter the following error:
KeyError: 'qwen2_moe'.
We do not advise you to use base language models for text generation. Instead, you can apply post-training, e.g., SFT, RLHF, continued pretraining, etc., on this model.
From the published model card. Full card on the HuggingFace links in the sidebar.
Using it via the API
Once AxForge deploys qwen1-5-moe-a for you, it answers on the OpenAI-compatible API — the same base URL and keys as every other model. (qwen1-5-moe-a below is illustrative; you get the exact model name on deployment.)
$ curl -sS https://api.axforge.ai/v1/chat/completions \
-H "Authorization: Bearer $AXFORGE_API_KEY" \
-H "Content-Type: application/json" \
-d '{"model":"qwen1-5-moe-a","messages":[{"role":"user","content":"Hello"}]}'
Create an account — your API key is available in the console. 3M free tokens every 30 days with every new account.