Nemotron-Cascade-2

Benchmark Results

• † Numbers in brackets refers to Tool-Integrated Reasoning (TIR) results.
• ‡ For the baseline models, we evaluate them using the recommended settings when official numbers are unavailable; otherwise, we report the official numbers directly.

Quick Start

• Nemotron-Cascade-2-30B-A3B follows the ChatML template and supports both thinking and instruct (non-thinking) modes. Reasoning content is enclosed within and tags. To activate the instruct (non-thinking) mode, we prepend `` to the beginning of the assistant’s response.

• Nemotron-Cascade-2-30B-A3B supports up to a 1M-token context length.

• Nemotron-Cascade-2-30B-A3B does not currently support OpenCode; it primarily supports OpenHands for agentic coding and SWE tasks.

• To reduce the context length in a multi-turn conversation, when the previous user turn involves thinking mode, only the final summary of the model's output will be added to the conversation history.

• Note that we do not define a separate tool role for tool responses; instead, we place them under the user role and warp them with and.

• We recommend setting the sampling parameters to temperature = 1.0 and top_p = 0.95.

vLLM setup

Requires vLLM version >= 0.17.1. The following will create API endpoints at http://localhost:8000/v1:

• Standard version: Use the following command to create an API endpoint with a maximum context length of 1M tokens.

  vllm serve nvidia/Nemotron-Cascade-2-30B-A3B --port 8000 --tensor-parallel-size 1 --gpu-memory-utilization 0.9 --max-model-len 262144 --reasoning-parser nano_v3 --mamba-ssm-cache-dtype float32 --port 8000 --trust_remote_code
  

• Tool Call: Use the following command to enable tool support.

  vllm serve nvidia/Nemotron-Cascade-2-30B-A3B --port 8000 --tensor-parallel-size 1 --gpu-memory-utilization 0.9 --max-model-len 262144 --reasoning-parser nano_v3 --mamba-ssm-cache-dtype float32 --port 8000 --trust_remote_code --enable-auto-tool-choice --tool-call-parser qwen3_coder
  

UPDATE (July 9, 2026): We changed --reasoning-parser from nemotron_v3 to nano_v3. The nano_v3 parser simply splits the output on : content before is treated as reasoning content, and content after it is treated as summary content. If is not present, the entire output remains in `reasoning content`. In contrast, the `nemotron_v3` parser treats or as an implicit end-of-reasoning boundary when is missing. This difference affects tool-calling scenarios.

Chat Template

from transformers import AutoTokenizer

model_name = 'nvidia/Nemotron-Cascade-2-30B-A3B'
tokenizer = AutoTokenizer.from_pretrained(model_name)

'''
single-turn example
'''
messages = [
  {"role": "system", "content": "You are a helpful and harmless assistant.\n\nYou are not allowed to use any tools"},
  {"role": "user", "content": "calculate 1+1?"}
]

# thinking mode
prompt_thinking = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True, enable_thinking=True)
# prompt_thinking = 'system\nYou are a helpful and harmless assistant.\n\nYou are not allowed to use any tools\nuser\ncalculate 1+1?\nassistant\n\n'

# instruct mode
prompt_instruct = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True, enable_thinking=False)
# prompt_instruct = 'system\nYou are a helpful and harmless assistant.\n\nYou are not allowed to use any tools\nuser\ncalculate 1+1?\nassistant\n'

'''
multi-turn example
'''
messages = [
    {"role": "system", "content": "You are a helpful and harmless assistant.\n\nYou are not allowed to use any tools"},
    {"role": "user", "content": "calculate 1+1?"},
    {"role": "assistant", "content": "THINKING_CONTENT\nTo calculate \\(1 + 1\\):\n\n1. **Identify the operation**: This is a basic addition problem involving two integers.\n2. **Perform the addition**:  \n   \\(1 + 1 = 2\\).\n\n**Result**: \\(\\boxed{2}\\)",},
    {"role": "user", "content": "what about 2+2"}
]

# thinking mode
prompt_thinking = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True, enable_thinking=True)
# prompt_thinking = 'system\nYou are a helpful and harmless assistant.\n\nYou are not allowed to use any tools\nuser\ncalculate 1+1?\nassistant\n\nTo calculate \\(1 + 1\\):\n\n1. **Identify the operation**: This is a basic addition problem involving two integers.\n2. **Perform the addition**:  \n   \\(1 + 1 = 2\\).\n\n**Result**: \\(\\boxed{2}\\)\nuser\nwhat about 2+2\nassistant\n\n'

# instruct mode
prompt_instruct = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True, enable_thinking=False)
# prompt_instruct = 'system\nYou are a helpful and harmless assistant.\n\nYou are not allowed to use any tools\nuser\ncalculate 1+1?\nassistant\n\nTo calculate \\(1 + 1\\):\n\n1. **Identify the operation**: This is a basic addition problem involving two integers.\n2. **Perform the addition**:  \n   \\(1 + 1 = 2\\).\n\n**Result**: \\(\\boxed{2}\\)\nuser\nwhat about 2+2\nassistant\n'

Python Tool Use