chandra-ocr-2

Chandra 2 is a state of the art OCR model from Datalab that outputs markdown, HTML, and JSON. It is highly accurate at extracting text from images and PDFs, while preserving layout information.

Try Chandra in the free playground, or use the hosted API for higher accuracy and speed.

What's New in Chandra 2

• 85.8% olmocr bench score (sota), 77.8% multilingual bench score (12% improvement over Chandra 1)
• Significant improvements to math, tables, complex layouts
• Improved layout, especially on wider documents
• Significantly better image captioning
• 90+ language support with major accuracy gains

Features

• Convert documents to markdown, HTML, or JSON with detailed layout information
• Excellent handwriting support
• Reconstructs forms accurately, including checkboxes
• Strong performance with tables, math, and complex layouts
• Extracts images and diagrams, with captions and structured data
• Support for 90+ languages

Quickstart

pip install chandra-ocr

# With vLLM (recommended, easy install)
chandra_vllm
chandra input.pdf ./output

# With HuggingFace (requires torch)
pip install chandra-ocr[hf]
chandra input.pdf ./output --method hf

Usage

With vLLM (recommended)

from chandra.model import InferenceManager
from chandra.model.schema import BatchInputItem
from PIL import Image

# Start vLLM server first with: chandra_vllm
manager = InferenceManager(method="vllm")
batch = [
    BatchInputItem(
        image=Image.open("document.png"),
        prompt_type="ocr_layout"
    )
]
result = manager.generate(batch)[0]
print(result.markdown)

With HuggingFace Transformers

from transformers import AutoModelForImageTextToText, AutoProcessor
from chandra.model.hf import generate_hf
from chandra.model.schema import BatchInputItem
from chandra.output import parse_markdown
from PIL import Image
import torch

model = AutoModelForImageTextToText.from_pretrained(
    "datalab-to/chandra-ocr-2",
    dtype=torch.bfloat16,
    device_map="auto",
)
model.eval()
model.processor = AutoProcessor.from_pretrained("datalab-to/chandra-ocr-2")
model.processor.tokenizer.padding_side = "left"

batch = [
    BatchInputItem(
        image=Image.open("document.png"),
        prompt_type="ocr_layout"
    )
]

result = generate_hf(batch, model)[0]
markdown = parse_markdown(result.raw)
print(markdown)

Benchmarks

olmOCR Benchmark

Examples