OLMo-2-1124

About

What OLMo-2-1124 is

OLMo-2-1124-7B-Instruct NOTE: 1/3/2025 UPDATE: Upon the initial release of OLMo-2 models, we realized the post-trained models did not share the pre-tokenization logic that the base models use. As a result, we have trained new post-trained models. The new models are available under the same names as the original models, but we have made the old models available with a postfix "-preview". See OLMo 2 Preview Post-trained Models for the colleciton of the legacy models. Release Documentation OLMo 2 7B Instruct November 2024 is post-trained variant of the OLMo-2 7B November 2024 model, which has undergone supervised finetuning on an OLMo-specific variant of the Tülu 3 dataset and further DPO training on this dataset, and finally RLVR training using this data. Tülu 3 is designed for state-of-the-art performance on a diversity of tasks in addition to chat, such as MATH, GSM8K, and IFEval. Check out the OLMo 2 paper or Tülu 3 paper for more details! OLMo is a series of Open Language Models designed to enable the science of language models. These models are trained on the Dolma dataset. We are releasing all code, checkpoints, logs (coming soon), and associated training details. The core models released in this batch include the following: Model description - Model type: A model trained on a mix of publicly available, synthetic and human-created datasets. - Language(s) (NLP): Primarily English - License: Apache 2.0 - Finetuned from model: allenai/OLMo-2-7B-1124-DPO Model Sources - Project Page: https://allenai.org/olmo - Repositories: - Core repo (training, inference, fine-tuning etc.): https://github.com/allenai/OLMo - Evaluation code: https://github.com/allenai/olmes - Further fine-tuning code: https://github.com/allenai/open-instruct - Paper: https://arxiv.org/abs/2501.00656 - Demo: https://playground.allenai.org/ Installation OLMo 2 will be supported in the next version of Transformers, and you need to install it from the main branch using: Using the model Loading with HuggingFace To load the model with HuggingFace, use the following snippet: Chat template The chat template for our models is formatted as: Or with new lines expanded: It is embedded within the tokenizer

Summarised from the published model card. Read the full card on the HuggingFace links below.

Specifications

What it is

How it works

How language models work

Variants

Sizes & precisions

Open weights ship in several sizes and precisions. One page, all the variants — pick the one that fits your GPU. VRAM figures are estimates from model size.

Using it via the API

Call it like any OpenAI endpoint

Once AxForge deploys olmo-2-1124 for you, it answers on the OpenAI-compatible API — the same base URL and keys as every other model. (olmo-2-1124 below is illustrative; you get the exact model name on deployment.)

$ curl -sS https://api.axforge.ai/v1/chat/completions \
  -H "Authorization: Bearer $AXFORGE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"olmo-2-1124","messages":[{"role":"user","content":"Hello"}]}'

Details

Languages, data & research

Papers

Licence

Open weights

Sources

Weights & code

Explore

More language models