HunyuanOCR

🤗 HF Model | 💻 GitHub Repo | 📄 Paper

📦 Model layout. This repository hosts HunyuanOCR-1.5 checkpoint at the root (target base weights). The DFlash speculative-decoding draft lives under dflash/, and the previous HunyuanOCR-1.0 is archived under v1.0/ (load it with subfolder="v1.0", or download the v1.0/ directory directly).

📖 Introduction

HunyuanOCR-1.5 is a lightweight, end-to-end OCR-specialized vision-language model. It targets a broad range of text-centric visual tasks and unifies document parsing, text spotting, information extraction, text-image translation within a single end-to-end VLM.

Building upon the validated lightweight architecture of HunyuanOCR-1.0, HunyuanOCR-1.5 does not redesign the model backbone. Instead, it performs a systematic upgrade around two goals — making the model faster and better:

• ⚡ Faster — DFlash inference acceleration. End-to-end OCR is often accompanied by long autoregressive decoding, which becomes the major bottleneck for dense documents, tables, formulas, and other long structured outputs. HunyuanOCR-1.5 adapts a speculative-decoding framework based on DFlash: a lightweight block-diffusion draft model drafts multiple candidate tokens in parallel, which are then verified by the target model in a single pass. This significantly reduces the decoding latency of long structured outputs while preserving the output distribution of the target model.
• 💻 PC-side deployment via llama.cpp. Beyond server-grade vLLM, HunyuanOCR-1.5 also supports CPU / consumer-GPU / laptop deployment through llama.cpp with a GGUF-converted checkpoint and an OpenAI-compatible llama-server. A DFlash-adapted llama.cpp fork is provided as well, so the same speculative-decoding acceleration is available on PC.
• 🧠 Better — Agentic Data Flow + upgraded training recipe. On the data side, we propose Agentic Data Flow, an agent-driven data-construction system that translates model weaknesses into executable data requirements. Agents deeply participate in material search, tool-based verification, sample cleaning, and data-pipeline development, and iterate in a closed loop with algorithm engineers. In HunyuanOCR-1.5, this system is used for targeted long-tail capabilities such as low-resource OCR, ancient-script OCR, and multi-image text-centric QA. On the training side, we systematically upgrade the recipe: pretraining Stage-3 is re-planned to incorporate the newly produced capability data, multi-image data, and historical OCR data, with maximum image resolution extended to 4K and context window extended to 128K; post-training refines the SFT data and further explores RL across different OCR tasks to amplify the gains from reinforcement learning.

Together, HunyuanOCR-1.5 achieves both faster inference and broader OCR capability coverage while retaining the deployment advantages of a lightweight end-to-end model. The full SFT / DFlash training pipeline and the transformers / vLLM / llama.cpp inference stack are open-sourced in the GitHub repo.

⚙️ Environment

Inference now uses a single unified environment (built on uv, requires CUDA 13) that runs all three configurations from the same install: vLLM AR, DFlash speculative decoding, and native transformers. Accuracy alignment across the three has been verified.

pip install uv
uv venv --python 3.12.11 && source .venv/bin/activate
uv pip install "vllm>=0.25.1"
uv pip install --no-build-isolation --no-cache-dir "flash-attn==2.8.3"

The inference code lives on GitHub under inference/ (inference/vLLM, inference/DFlash, inference/transformers). See docs/inference/inference.md for the full setup and usage. If you lack CUDA 13 or only need one configuration, that document also points to the lighter per-configuration recipes in the archive.

Common prerequisites: Python 3.10+ (3.12 tested), an NVIDIA GPU, and huggingface_hub for downloading the weights:

pip install -U "huggingface_hub[cli]"
# target base (1.5) — skip the archived 1.0 to save space
huggingface-cli download tencent/HunyuanOCR --local-dir ./HunyuanOCR --exclude "v1.0/*"

The download contains both the base model and the dflash/ draft model.

🧪 Inference

All configurations share the same weights and the same task-type prompts + sampling (temperature=0.0, top_p=1.0, top_k=-1, repetition_penalty=1.08) + post-processing, so their outputs are directly comparable. Grab the toolkit from GitHub first:

git clone https://github.com/Tencent-Hunyuan/HunyuanOCR.git
cd HunyuanOCR

A. HuggingFace transformers (native)

The model ships the official HunYuanVLForConditionalGeneration + AutoProcessor integration (transformers ≥ 5.13.0). The simplest path — weights are pulled from the Hub automatically:

import torch
from transformers import AutoProcessor, HunYuanVLForConditionalGeneration

MODEL_ID = "tencent/HunyuanOCR"

processor = AutoProcessor.from_pretrained(MODEL_ID, trust_remote_code=True, use_fast=False)
model = HunYuanVLForConditionalGeneration.from_pretrained(
    MODEL_ID, torch_dtype=torch.bfloat16, device_map="auto",
    trust_remote_code=True,
).eval()

prompt = (
    "提取文档图片中正文的所有信息用markdown格式表示，其中页眉、页脚部分忽略，"
    "表格用html格式表达，文档中公式用latex格式表示，按照阅读顺序组织进行解析。"
)
messages = [{
    "role": "user",
    "content": [
        {"type": "image", "image": "/path/to/document.png"},
        {"