hayai-ocr

About

What hayai-ocr is

Hayai OCR v2.1 Hayai is a lightweight (~150M parameter) vision-to-text OCR model designed for fast, crop-level transcription across Japanese, Chinese, Korean, and English. By pairing Google’s SigLIP2 NaFlex vision encoder with a 12-layer custom causal transformer decoder, Hayai reads dense, stylized, horizontal, and vertical text directly from images in a single forward pass without requiring a separate text detection stage (e.g., DBNet/YOLO). (Doesn't work for full pages though. Only crops) What's New in v2.1 (Joint Multimodal + Linguistic Pretraining) In prior versions, compact OCR models struggled with visually ambiguous CJK radicals and homoglyphs (e.g., confusing 銀 vs. 高 or 校 vs. 枚) because a pure image-trained decoder lacked statistical language priors. Hayai v2.1 introduces Joint Multi-Task Training: - Zero-Overhead Language Prior Injection: Co-trained directly on streaming Wikipedia (JA, ZH, KO, EN) and Aozora Bunko corpora. Text-only passes enter the decoder directly with 1D RoPE (bypassing the vision backbone), teaching the decoder deep contextual CJK transition probabilities. - Radical & Counter Disambiguation: Eliminates homograph and counter errors on complex layouts. - Flawless Multi-Script Support: Significantly boosted English accuracy while maintaining high precision on vertical Japanese, Korean Hangul, and Chinese Hanzi. Architecture - Total Parameters: ~150M - Vision Encoder: google/siglip2-base-patch16-naflex (~86M params) - Native aspect ratio preservation via NaFlex patching (no forced warping/squashing of dense characters). - Projector: 2-layer MLP mapping visual patch features into the decoder hidden dimension. - Decoder: 12-layer Causal Transformer (~60M params) - Attention: Grouped-Query Attention (8 query heads, 2 key/value heads) with RMSNorm on queries & keys. - FFN: SwiGLU feed-forward layers (dmodel = 512, dffn = 2048). - Positional Embeddings: Dynamic 2D Multimodal Rotary Position Embeddings (2D mRoPE) over visual tokens; 1D RoPE over text tokens. - Attention Masking: Block-causal attention (bidirectional among visual patch tokens, causal across output text tokens). Usage Note: trustremotecode=True is required because the model ut

Summarised from the published model card. Read the full card on the HuggingFace links below.

Specifications

What it is

How it works

How language models work

Variants

Sizes & precisions

Open weights ship in several sizes and precisions. One page, all the variants — pick the one that fits your GPU. VRAM figures are estimates from model size.

Using it via the API

Call it like any OpenAI endpoint

Once AxForge deploys hayai-ocr for you, it answers on the OpenAI-compatible API — the same base URL and keys as every other model. (hayai-ocr below is illustrative; you get the exact model name on deployment.)

$ curl -sS https://api.axforge.ai/v1/chat/completions \
  -H "Authorization: Bearer $AXFORGE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"hayai-ocr","messages":[{"role":"user","content":"Hello"}]}'

Details

Languages, data & research

Licence

Open weights

Sources

Weights & code

Explore

More language models