Model reference · open weights
Nous-Hermes-2-Mixtral-8x-SFT is an open-weight language model from NousResearch. AxForge deploys and operates it for you on dedicated EU-owned hardware — with the licence handled where one is required.
Available as managed deployment — configured and operated for you on dedicated EU hardware, quoted per deployment.
What it is
| Maker | NousResearch |
|---|---|
| Type | Language models |
| Task | Text gen |
| Parameters (lead) | 46.7B |
| Context | 32k tokens |
| Runs with | transformers |
| Based on | mistralai/Mixtral-8x7B-v0.1 |
| Released | 2023-12-26 |
| Popularity | 4k downloads / month |
| Licence | Open weights |
About
Nous Hermes 2 Mixtral 8x7B SFT is the supervised finetune only version of our new flagship Nous Research model trained over the Mixtral 8x7B MoE LLM.
The model was trained on over 1,000,000 entries of primarily GPT-4 generated data, as well as other high quality data from open datasets across the AI landscape, achieving state of the art performance on a variety of tasks.
This is the SFT only version of Mixtral Hermes 2, we have also released an SFT+DPO version, for people to find which works best for them, which can be found here: https://huggingface.co/NousResearch/Nous-Hermes-2-Mixtral-8x7B-DPO
Nous-Hermes 2 on Mixtral 8x7B SFT is the bedrock for major improvements on many of the benchmarks below compared to the base Mixtral model, and is the SFT only version of our first model to beat the flagship Mixtral Finetune by MistralAI (the DPO version).
| Task |Version| Metric |Value | |Stderr|
|-------------|------:|--------|-----:|---|-----:|
|arc_challenge| 0|acc |0.5904|± |0.0144|
| | |acc_norm|0.6323|± |0.0141|
|arc_easy | 0|acc |0.8594|± |0.0071|
| | |acc_norm|0.8607|± |0.0071|
|boolq | 1|acc |0.8783|± |0.0057|
|hellaswag | 0|acc |0.6592|± |0.0047|
| | |acc_norm|0.8434|± |0.0036|
|openbookqa | 0|acc |0.3400|± |0.0212|
| | |acc_norm|0.4660|± |0.0223|
|piqa | 0|acc |0.8324|± |0.0087|
| | |acc_norm|0.8379|± |0.0086|
|winogrande | 0|acc |0.7569|± |0.0121|
Average: 75.36
| Task |Version| Metric |Value | |Stderr|
|------------------------------|------:|--------|-----:|---|-----:|
|agieval_aqua_rat | 0|acc |0.2441|± |0.0270|
| | |acc_norm|0.2598|± |0.0276|
|agieval_logiqa_en | 0|acc |0.4025|± |0.0192|
| | |acc_norm|0.3978|± |0.0192|
|agieval_lsat_ar | 0|acc |0.2391|± |0.0282|
| | |acc_norm|0.2043|± |0.0266|
|agieval_lsat_lr | 0|acc |0.5353|± |0.0221|
| | |acc_norm|0.5098|± |0.0222|
|agieval_lsat_rc | 0|acc |0.6617|± |0.0289|
| | |acc_norm|0.5948|± |0.0300|
|agieval_sat_en | 0|acc |0.7961|± |0.0281|
| | |acc_norm|0.7816|± |0.0289|
|agieval_sat_en_without_passage| 0|acc |0.4757|± |0.0349|
| | |acc_norm|0.4515|± |0.0348|
|agieval_sat_math | 0|acc |0.4818|± |0.0338|
| | |acc_norm|0.3909|± |0.0330|
Average: 44.89
| Task |Version| Metric |Value | |Stderr|
|------------------------------------------------|------:|---------------------|-----:|---|-----:|
|bigbench_causal_judgement | 0|multiple_choice_grade|0.5789|± |0.0359|
|bigbench_date_understanding | 0|multiple_choice_grade|0.7154|± |0.0235|
|bigbench_disambiguation_qa | 0|multiple_choice_grade|0.5388|± |0.0311|
|bigbench_geometric_shapes | 0|multiple_choice_grade|0.4680|± |0.0264|
| | |exact_str_match |0.0000|± |0.0000|
|bigbench_logical_deduction_five_objects | 0|multiple_choice_grade|0.3260|± |0.0210|
|bigbench_logical_deduction_seven_objects | 0|multiple_choice_grade|0.2443|± |0.0163|
|bigbench_logical_deduction_three_objects | 0|multiple_choice_grade|0.5233|± |0.0289|
|bigbench_movie_recommendation | 0|multiple_choice_grade|0.3700|± |0.0216|
|bigbench_navigate | 0|multiple_choice_grade|0.5000|± |0.0158|
|bigbench_reasoning_about_colored_objects | 0|multiple_choice_grade|0.6665|± |0.0105|
|bigbench_ruin_names | 0|multiple_choice_grade|0.6317|± |0.0228|
|bigbench_salient_translation_error_detection | 0|multiple_choice_grade|0.2505|± |0.0137|
|bigbench_snarks | 0|multiple_choice_grade|0.7127|± |0.0337|
|bigbench_sports_understanding | 0|multiple_choice_grade|0.6592|± |0.0151|
|bigbench_temporal_sequences | 0|multiple_choice_grade|0.6860|± |0.0147|
|bigbench_tracking_shuffled_objects_five_objects | 0|multiple_choice_grade|0.2200|± |0.0117|
|bigbench_tracking_shuffled_objects_seven_objects| 0|multiple_choice_grade|0.1503|± |0.0085|
|bigbench_tracking_shuffled_objects_three_objects| 0|multiple_choice_grade|0.5233|± |0.0289|
Average: 48.69
Nous Hermes 2 uses ChatML as the prompt format, opening up a much more structured system for engaging the LLM in multi-turn chat dialogue.
System prompts allow steerability and interesting new ways to inte
From the published model card. Full card on the HuggingFace links in the sidebar.
Using it via the API
Once AxForge deploys nous-hermes-2-mixtral-8x-sft for you, it answers on the OpenAI-compatible API — the same base URL and keys as every other model. (nous-hermes-2-mixtral-8x-sft below is illustrative; you get the exact model name on deployment.)
$ curl -sS https://api.axforge.ai/v1/chat/completions \
-H "Authorization: Bearer $AXFORGE_API_KEY" \
-H "Content-Type: application/json" \
-d '{"model":"nous-hermes-2-mixtral-8x-sft","messages":[{"role":"user","content":"Hello"}]}'
Create an account — your API key is available in the console. 5M tokens/month currently included with every new account at launch.