ZGCM-1

Introduction

ZGCM-1 is a 7.39B-parameter dense language model trained from scratch, built for mathematical reasoning and tool-assisted search. It combines deliberate internal thinking with active information gathering, supporting 256K-token context and both thinking and direct-response modes in a single model.

The project brings together an efficient hybrid-attention architecture, FP8 training with Muon, progressive long-context mid-training, and general-agentic supervised fine-tuning. Researcher-directed AI agents contribute throughout development, from data curation and cluster operations to evaluation and deployment.

Highlights

• Math and reasoning at 7B scale. ZGCM-1 achieves 97.13% on MATH-500, 75.00% on AIME 2026, and 70.42% on HMMT 2025, with the best average rank across the report's 14 reasoning benchmarks among the seven compared 7B–8B models.
• Search beyond model memory. Multi-step tool use reaches 63.09% on WebWalkerQA, 19.43% on BrowseComp, and 62.00% on Binary Function Search.
• Efficient long context. Gated sliding-window and global attention deliver 3.94× training throughput at 256K compared with full attention in the report's architecture experiments. The report estimates an approximately 4.2× improvement in 16K pretraining time-to-loss from combined architecture, precision, optimizer, and normalization gains.
• An open research recipe. The training repository releases the data-processing, pretraining, mid-training, and SFT workflows, with stage-specific configurations and runtime documentation.

Quickstart

ZGCM-1 ships its own modeling code, so trust_remote_code=True is required.

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "zgcagi/ZGCM-1-7B"
tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    trust_remote_code=True,
    dtype="bfloat16",
    device_map="auto",
)

messages = [{"role": "user", "content": "Compute 1+1."}]
prompt = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
    enable_thinking=True,
)
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(
    **inputs,
    max_new_tokens=4096,
    do_sample=True,
    temperature=1.0,
    top_p=1.0,
)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))

Evaluation Results

The following results are from the technical report, using the 256K SFT checkpoint in thinking mode. Non-agentic evaluations use temperature 1.0, top-p 1.0, and mean pass@1 over 32 runs unless otherwise specified.

Selected reasoning benchmarks

Selected rows and models from Table 2; bold marks the best score in each displayed row. The full evaluation covers 20 benchmarks, including code, knowledge, and instruction following.

Agentic search

Source: Tables 3–4. Web research allows up to 64 search-and-read steps. Binary Function Search uses a separate protocol on 50 tasks from 10 held-out projects.

Architecture and Training

The training recipe described in the report has three main stages:

1. Pretraining: approximately 4.19T tokens, combining curriculum-based data mixing with hybrid FP8 precision and Muon optimization.
2. Mid-training: approximately 600B tokens with context extended from 16K → 64K → 256K. Interaction traces are reformulated as Markov Decision Process (MDP) state-action transitions to supervise individual decisions.
3. Supervised fine-tuning: joint general and agentic training with mixed thinking/direct-response examples, execution-verified trajectories, and assistant-only loss. The report also explores mixed RL for mathematics and code.

Resources

For dataset usage, see the ZGCM-1-Data card. Model specifications, evaluation details, and figures are presented in *ZGCM-1: A Fully Open and Extremely Efficient Foundation Model for Math and Agentic Search