granite-guardian-4.1

What's New

Granite Guardian 4.1 8B introduces improved Bring Your Own Criteria (BYOC) support, enabling users to define arbitrary judging criteria beyond the pre-baked safety and hallucination detectors. The model can now faithfully evaluate complex, multi-part requirements such as formatting rules, length constraints, and domain-specific instructions.

Key improvements over Granite Guardian 3.3:

• BYOC capability: Large gains on instruction-following benchmarks. For example, IFEval multi-constraint BAcc improves from 0.458 to 0.844 (no-think), InfoBench (Human) from 0.535 to 0.706, InfoBench (GPT-4) from 0.585 to 0.726.
• Best-of-N reward model: When used as a reward model for best-of-N selection on the verifiable tasks in the JETTS benchmark, Granite Guardian 4.1 8B achieves an overall score of 70.29, outperforming all tested reward models up to 70B parameters.
• Hybrid thinking: Supports both thinking mode (with detailed reasoning traces) and non-thinking mode (low-latency yes/no judgements).
• Function calling: Stronger hallucination detection in agentic workflows: BAcc improves from 0.74 to 0.79 (no-think).
• Maintained safety and groundedness: Competitive with prior releases on OOD safety (F1 0.79 no-think) and RAG groundedness (Avg BAcc 0.76 think).

Model Summary

Granite Guardian 4.1 8B is a specialized safety model fine-tuned from ibm-granite/granite-4.1-8b, designed to judge if the input prompts and the output responses of an LLM-based system meet specified criteria. The model comes pre-baked with certain criteria including but not limited to: jailbreak attempts, profanity, and hallucinations related to tool calls and retrieval augmented generation in agent-based systems. Additionally, the model also allows users to bring their own criteria and tailor the judging behavior to specific use cases.

This version of Granite Guardian is a hybrid thinking model that allows the user to operate in thinking or non-thinking mode. In thinking mode, the model produces detailed reasoning traces through ... and ... tags. In non-thinking mode, the model only produces the judgement score through the ... tags.

It is trained on unique data comprising human annotations and synthetic data informed by internal red-teaming. It outperforms other open-source models in the same space on standard benchmarks.

• Developers: IBM Research
• GitHub Repository: ibm-granite/granite-guardian
• Cookbook: Granite Guardian Recipes
• Website: Granite Guardian Docs
• Paper: Granite Guardian
• Release Date: April, 2026
• License: Apache 2.0

Usage

Intended Use: The guardian model must be used strictly for the prescribed scoring mode, which generates yes/no outputs based on the specified template. Any deviation from this intended use may lead to unexpected, potentially unsafe, or harmful outputs. The model may also be prone to such behavior via adversarial attacks.

Installation

pip install transformers torch vllm

Prompting Guide

Granite Guardian uses a structured prompt format to instruct the model to judge text against a given criterion. Understanding how this block is built is essential for getting reliable results.

1. Think / No-Think Instruction

Every guardian block begins with a mode selector that controls whether the model produces reasoning traces before scoring:

• Think mode (``): The model reasons step-by-step inside ... tags before emitting a score. Use this when you want interpretable justifications.
• No-think mode (``): The model skips reasoning and outputs the score directly. Use this for lower-latency evaluation where only the yes/no judgement is needed.

The two instruction strings are:

# Think mode
given judging criteria. Use all available information, including conversations,
documents, and tools. Reason through the evidence step by step before providing
your score. Wrap your reasoning in  tags, then provide your
score in  tags.

# No-think mode
judging criteria using all available information, including conversations,
documents, and tools. Provide your score immediately without explanation.
Output empty \n tags followed by your score in  tags.

2. Criteria

After the mode instruction, the block includes a ### Criteria: section that tells the model what to judge. This can be:

• A pre-baked criterion, one of the built-in definitions (e.g., function calling hallucination, groundedness, jailbreaking). See the full list in the section below.
• A custom (BYOC) criterion, any user-defined rule expressed in natural language (e.g., "The response contains no more than 3 sentences.").

The criteria text is free-form; the model applies it as a binary yes/no test against the content in the conversation.

3. Scoring Schema

The block always ends with a hard-coded scoring instruction:

For prompt classification

### Scoring Schema: If the last user's text meets the criteria, return 'yes'; otherwise, return 'no'.

For response classification

### Scoring Schema: If the last assistant's text meets the criteria, return 'yes'; otherwise, return 'no'.

This ensures the model produces a deterministic binary output (yes or no) inside ... tags, regardless of mode.

Putting It Together

The complete guardian block is assembled as follows and appended as the final user message in the conversation:

### Criteria: {criteria_text}

### Scoring Schema: If the last user's/assistant's text meets the criteria, return 'yes'; otherwise, return 'no'.

List of Pre-baked Criteria {criteria_text}

The model is t