Apriel-H1-Thinker-SFT

A 15B-parameter hybrid reasoning model combining Transformer attention and Mamba State Space layers for high efficiency and scalability. Derived from Apriel-Nemotron-15B-Thinker through progressive distillation, Apriel-H1 replaces less critical attention layers with linear Mamba blocks—achieving over 2× higher inference throughput in vLLM with minimal loss in reasoning, math, and coding performance.

• Model Size: 15B parameters
• Context Length: 65K (target; runtime dependent)
• Languages: English (best)

Highlights

• Hybrid Transformer–SSM architecture
• ~2× throughput improvement over the base Thinker model
• Retains strong reasoning, math, and coding capabilities
• Built via efficient distillation—no training from scratch required

Model Overview

Apriel-H1-15b-Thinker is designed for agentic tasks, code assistance, and multi-step reasoning. It follows Apriel’s “think then answer” style: the model first produces a hidden chain-of-thought and then a concise final response. Where reasoning traces are undesired, configure prompts to favor concise outputs.

Technical report: Apriel-H1 Report

Efficient and strong among hybrids

All models were evaluated with vllm server endpoints using FlashInfer (except for AI21-Jamba-Reasoning-3B which used FlashAttention2), mamba_cache was set to fp32 for models: NVIDIA-Nemotron-Nano-9B-v2 and AI21-Jamba-Reasoning-3B.

Comparing with Thinker ~2x speedup!

How to Use

Install dependencies:

pip install transformers==4.53.2

Basic usage with Transformers generate:

import re
from transformers import AutoModelForCausalLM, AutoTokenizer

model_name = "ServiceNow-AI/Apriel-H1-15b-Thinker-SFT"

# load the tokenizer and the model
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(
    model_name,
    torch_dtype="auto",
    device_map="auto"
)

prompt = "Positive real numbers $x$ and $y$ satisfy $y^3=x^2$ and $(y-x)^2=4y^2$. What is $x+y$?\nMark your solution with \\boxed"
messages = [
    {"role": "user", "content": prompt}
]

text = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
    tools=[]
)
model_inputs = tokenizer([text], return_tensors="pt").to(model.device)

generated_ids = model.generate(**model_inputs, max_new_tokens=1024)
output = tokenizer.decode(generated_ids[0], skip_special_tokens=True)

response = re.findall(r"\\[BEGIN FINAL RESPONSE\\](.*?)\\[END FINAL RESPONSE\\]", output, re.DOTALL)[0].strip()
print("response:", response)

Recommended settings: temperature 0.6; increase max_new_tokens for complex reasoning.

Use it with vLLM

💻 Local Installation

1. Create and activate a Python environment

You can use any environment manager. The example below uses uv:

uv venv --python 3.12 --seed
source .venv/bin/activate

2. Install vLLM and the Apriel plugin

Find our plugin at https://github.com/ServiceNow/apriel. You may need to install a version of vLLM compatible with your CUDA version.

In this example, we use the default CUDA version and let vLLM automatically select the correct backend.

git clone git@github.com:ServiceNow/apriel.git
cd apriel
uv pip install vllm==0.10.2 --torch-backend=auto
pip install .

🧠 Running a vLLM Server

Option 1: Run locally (from source install)

Once installed, you can launch a vLLM OpenAI-compatible API server with your Apriel model:

vllm serve \
  --model ServiceNow-AI/Apriel-H1-15b-Thinker-SFT \
  --port 8000

Option 2: Run via Docker

You can run the server directly using the prebuilt container:

docker run --runtime nvidia --gpus all \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HUGGING_FACE_HUB_TOKEN=$HF_TOKEN" \
    -p 8000:8000 \
    --ipc=host \
    ghcr.io/servicenow/apriel:latest \
    --model ServiceNow-AI/Apriel-H1-15b-Thinker-SFT \

Chat Template

You are a thoughtful and systematic AI assistant built by ServiceNow Language Models (SLAM) lab. Before providing an answer, analyze the problem carefully and present your reasoning step by step. After explaining your thought process, provide the final solution in the following format: [BEGIN FINAL RESPONSE] ... [END FINAL RESPONSE].
# user message here
Here are my reasoning steps:
# thoughts here
[BEGIN FINAL RESPONSE]
# assistant response here
[END FINAL RESPONSE]

The model will first generate its thinking process and then generate its final response between [BEGIN FINAL RESPONSE] and [END FINAL RESPONSE]. Here is a code snippet demonstrating the application of the chat template:

from transformers import AutoTokenizer
model_name = "ServiceNow-AI/Apriel-H1-15b-Thinker-SFT"
tokenizer = AutoTokenizer.from_pretrained(model_name)

# prepare the model input
custom_system_prompt = "Answer like a pirate."
prompt = "You are an expert assistant in the implementation of customer experience management aspect of retail applications \n \nYou will be using Python as the programming language. \n \nYou will utilize a factory design pattern for the implementation and following the dependency inversion principle \n \nYou will modify the implementation based on user requirements. \n \nUpon user request, you will add, update, and remove the features & enhancements in the implementation provided by you. \n \nYou will ask whether the user wants to refactor the provided code or needs a sample implementation for reference. Upon user confirmation, I will proceed accordingly. \n \n**Guidelines:** \n 1. **User Requirements:** \n - You have to ask users about their requirements, clarify the user expectations, and suggest the best possible solution by providing examples of Python code snippets. \n - Ask users about which type of reports they need to as