Ovis2.5

Key Features

• Native-Resolution Perception — NaViT vision encoder preserves fine details and global structure without lossy tiling.
• Deep-Reasoning Capability — Optional thinking mode for self-checking and revision beyond linear CoT. Thinking budget supported.
• Chart & Document OCR — State-of-the-art at its scale for complex chart analysis, document understanding (including tables and forms), and OCR.
• Broad Task Coverage — Demonstrates leading performance on image reasoning, video understanding, and grounding benchmarks, showcasing strong general multimodal capability.

Quick Inference

Below is a simple example demonstrating how to run Ovis2.5 with a single image input. For accelerated inference with vLLM, refer to GitHub.

First, install the required dependencies:

pip install torch==2.4.0 transformers==4.51.3 numpy==1.25.0 pillow==10.3.0 moviepy==1.0.3
pip install flash-attn==2.7.0.post2 --no-build-isolation

Then, run the following code.

import torch
import requests
from PIL import Image
from transformers import AutoModelForCausalLM

MODEL_PATH = "AIDC-AI/Ovis2.5-2B"

# Thinking mode & budget
enable_thinking = True
enable_thinking_budget = True  # Only effective if enable_thinking is True.

# Total tokens for thinking + answer. Ensure: max_new_tokens > thinking_budget + 25
max_new_tokens = 3072
thinking_budget = 2048

model = AutoModelForCausalLM.from_pretrained(
    MODEL_PATH,
    torch_dtype=torch.bfloat16,
    trust_remote_code=True
).cuda()

messages = [{
    "role": "user",
    "content": [
        {"type": "image", "image": Image.open(requests.get("https://cdn-uploads.huggingface.co/production/uploads/658a8a837959448ef5500ce5/TIlymOb86R6_Mez3bpmcB.png", stream=True).raw)},
        {"type": "text", "text": "Calculate the sum of the numbers in the middle box in figure (c)."},
    ],
}]

input_ids, pixel_values, grid_thws = model.preprocess_inputs(
    messages=messages,
    add_generation_prompt=True,
    enable_thinking=enable_thinking
)
input_ids = input_ids.cuda()
pixel_values = pixel_values.cuda() if pixel_values is not None else None
grid_thws = grid_thws.cuda() if grid_thws is not None else None

outputs = model.generate(
    inputs=input_ids,
    pixel_values=pixel_values,
    grid_thws=grid_thws,
    enable_thinking=enable_thinking,
    enable_thinking_budget=enable_thinking_budget,
    max_new_tokens=max_new_tokens,
    thinking_budget=thinking_budget,
)

response = model.text_tokenizer.decode(outputs[0], skip_special_tokens=True)
print(response)

The thinking and thinking budget logic can be applied in the same way for multi-image, video and pure text scenarios.

Note (answer extraction for CoT/Thinking): To make evaluation and usage easier, we recommend appending a fixed suffix to prompts when using chain-of-thought (CoT) or thinking mode. This ensures the model clearly outputs a final answer that can be extracted programmatically:

End your response with 'Final answer: '.

For example:

Calculate the sum of the numbers in the middle box in figure (c).
End your response with 'Final answer: '.

Tip: The sections below include an optional streaming helper (compatible with two-phase thinking/budget runs) and extra inference modes: multi-image, video, and text-only.

To support thinking budget, we modified the implementation of the Ovis generate method and the default TextIteratorStreamer is now incompatible. If you need to stream model output, be sure to use the helper class below.

# --- Budget-aware streamer helper ---
from transformers import TextIteratorStreamer

class BudgetAwareTextStreamer(TextIteratorStreamer):
    """A streamer compatible with Ovis two-phase generation.

    Call .manual_end() after generation to flush any remaining text.
    """
    def manual_end(self):
        if len(self.token_cache) > 0:
            text = self.tokenizer.decode(self.token_cache, **self.decode_kwargs)
            printable_text = text[self.print_len:]
            self.token_cache = []
            self.print_len = 0
        else:
            printable_text = ""
        self.next_tokens_are_prompt = True
        self.on_finalized_text(printable_text, stream_end=True)

    # Disable base class's end hook; we'll finalize via manual_end()
    def end(self):
        pass

Example usage:

streamer = BudgetAwareTextStreamer(
    model.text_tokenizer,
    skip_prompt=True,
    skip_special_tokens=True
)

outputs = model.generate(
    inputs=input_ids,
    pixel_values=pixel_values,
    grid_thws=grid_thws,
    enable_thinking=enable_thinking,
    enable_thinking_budget=enable_thinking_budget,
    max_new_tokens=max_new_tokens,
    thinking_budget=thinking_budget,
    streamer=streamer
)

Demonstrates how to run inference with multiple images and a rel