Olmo-3.1-Think

We introduce Olmo 3, a new family of 7B and 32B models both Instruct and Think variants. Long chain-of-thought thinking improves reasoning tasks like math and coding.

Olmo is a series of Open language models designed to enable the science of language models. These models are pre-trained on the Dolma 3 dataset and post-trained on the Dolci datasets. We are releasing all code, checkpoints, logs (coming soon), and associated training details.

The core models released in this batch include the following:

Installation

Olmo 3 is supported in transformers 4.57.0 or higher:

pip install transformers>=4.57.0

Inference

You can use OLMo with the standard HuggingFace transformers library:

from transformers import AutoModelForCausalLM, AutoTokenizer
olmo = AutoModelForCausalLM.from_pretrained("allenai/Olmo-3.1-32B-Think")
tokenizer = AutoTokenizer.from_pretrained("allenai/Olmo-3.1-32B-Think")
message = ["Who would win in a fight - a dinosaur or a cow named Moo Moo?"]
inputs = tokenizer(message, return_tensors='pt', return_token_type_ids=False)
# optional verifying cuda
# inputs = {k: v.to('cuda') for k,v in inputs.items()}
# olmo = olmo.to('cuda')
response = olmo.generate(**inputs, max_new_tokens=100, do_sample=True, top_k=50, top_p=0.95)
print(tokenizer.batch_decode(response, skip_special_tokens=True)[0])
>> 'Okay, so the question is who would win in a fight...'

For faster performance, you can quantize the model using the following method:

AutoModelForCausalLM.from_pretrained("allenai/Olmo-3.1-32B-Think",
    torch_dtype=torch.float16,
    load_in_8bit=True)  # Requires bitsandbytes

The quantized model is more sensitive to data types and CUDA operations. To avoid potential issues, it's recommended to pass the inputs directly to CUDA using:

inputs.input_ids.to('cuda')

We have released checkpoints for these models. For post-training, the naming convention is step_XXX.

To load a specific model revision with HuggingFace, simply add the argument revision:

olmo = AutoModelForCausalLM.from_pretrained("allenai/Olmo-3.1-32B-Think", revision="step_300")

Or, you can access all the revisions for the models via the following code snippet:

from huggingface_hub import list_repo_refs
out = list_repo_refs("allenai/Olmo-3.1-32B-Think")
branches = [b.name for b in out.branches]

Chat template

Default System Message

The default system prompt for this model is:

You are a helpful AI assistant.

Chat Format

The chat template for this model is formatted as:

You are a helpful AI assistant.
Who would win in a fight - a dinosaur or a cow named Moo Moo?
Hmm, first I need to break this down. Let me think about the different factors involved here.....
Moo Moo the cow would certinaly win.

Model Description

• Developed by: Allen Institute for AI (Ai2)
• Model type: a Transformer style autoregressive language model.
• Language(s) (NLP): English
• License: This model is licensed under Apache 2.0. It is intended for research and educational use in accordance with Ai2's Responsible Use Guidelines.
• Contact: Technical inquiries: olmo@allenai.org. Press: press@allenai.org
• Date cutoff: Dec. 2024.

Model Sources

• Project Page: https://allenai.org/olmo
• Repositories:
  • Open-Instruct for DPO and RLVR: https://github.com/allenai/open-instruct
  • OLMo-Core for pre-training and SFT: https://github.com/allenai/OLMo-core
  • OLMo-Eval for evaluation: https://github.com/allenai/OLMo-Eval
• Paper: [TBD]

Evaluation