hayai-ocr-nova

What’s New in v2.5

Hayai v2.5 retains the core backbone of v2.1 (SigLIP2 NaFlex ~86M + 12-layer GQA decoder) while introducing major structural and efficiency upgrades:

• 4x Token Reduction (DSC Projector): Features a new Downsampling Spatial Convolution (DSCProjector) that reshapes patch embeddings into a 2D grid and applies a 2x pixel unshuffle (4 patches -> 1 token).
• Lower Prefill Latency & KV Footprint: By reducing decoder vision tokens by 75%, v2.5 dramatically cuts prefill latency and memory footprint, making higher patch budgets computationally affordable.
• Learnable Residual Scaling: Decoder layers now incorporate learnable per-channel scaling parameters (attn_res_scale, ffn_res_scale, initialized at 1.0) to stabilize deep residual propagation.
• Engineered for High-Throughput Inference: generate() features static KV-cache allocation, precomputed 1D text RoPE, and FP16 autocasting on CUDA devices.
• Auxiliary IDS Co-training: Pre-trained with an auxiliary 226-class Ideographic Description Sequence (IDS) classification head to sharpen character-level discrimination across rare CJK glyphs (training only; zero overhead at inference).

Architectural Comparison

Quickstart

import torch
from PIL import Image
from transformers import AutoModel, AutoProcessor, PreTrainedTokenizerFast

MODEL_ID = "JustANormalTinkerer/hayai-ocr-v2.5-nova"

model = AutoModel.from_pretrained(MODEL_ID, trust_remote_code=True).cuda().eval()
tokenizer = PreTrainedTokenizerFast.from_pretrained(MODEL_ID)
processor = AutoProcessor.from_pretrained("google/siglip2-base-patch16-naflex")

image = Image.open("example.png").convert("RGB")

# Select patch budget: 256 (throughput), 384 (balanced), or 512 (quality)
inputs = processor(images=[image], max_num_patches=384, return_tensors="pt").to("cuda")

with torch.no_grad():
    texts = model.generate(
        pixel_values=inputs["pixel_values"],
        pixel_attention_mask=inputs["pixel_attention_mask"],
        spatial_shapes=inputs["spatial_shapes"],
        tokenizer=tokenizer,
        max_new_tokens=128,
        repetition_penalty=1.0,  # Keep at 1.0 (disabled) for OCR
    )

print(texts[0])

Requirements: trust_remote_code=True is required for custom block-causal attention and 2D multi-dimensional RoPE (mRoPE). Standard greedy decoding is recommended. For helper utilities, see hayai-ocr on GitHub.

Resolution Strategy (max_num_patches)

Because SigLIP2 NaFlex scales inputs dynamically preserving aspect ratio, max_num_patches governs your spatial budget. Thanks to the 4x patch compression in v2.5, higher patch counts run significantly faster than in earlier architectures:

*Evaluated on JMangaBench_Mixed (3,286 crops) under NFKC normalization. Latencies measured on an NVIDIA T4 GPU.

Which patch budget should you use?

• 256 — Throughput-Oriented: Best for high-volume pipelines, wide-aspect crops, and clean horizontal text.
• 384 — Throughput-Oriented but still want quality: Excellent balance of speed and recognition fidelity. Closes over 70% of the accuracy gap to 512.
• 512 — Fine-Grained / Small Glyphs (Recommended): Critical for dense panels, small stylized fonts, and complex vertical layouts (e.g., dense slices drop CER from 7.43% down to 2.70%).

Benchmarks

Crop-Level Recognition (JMangaBench_Mixed)

Evaluated across 3,286 standard benchmark crops under Unicode NFKC normalization:

Block-Level E