Model reference · open weights
olmOCR-0825 is an open-weight language model from allenai. olmOCR-7B-0825-FP8 (FP8) weighs 10.1 GB; the smallest configuration that runs it is RTX 4060 Ti 16 GB.
Summary of the allenai/olmOCR-7B-0825-FP8 model card, 2026-10-01
What it is
| Released by | allenai |
|---|---|
| Released | 2025-08-13 |
| Parameters | 8.3B |
| VRAM | 10.1 GB for the weights |
What it runs on
| Card | Requests at once | Context max | Memory | |
|---|---|---|---|---|
| 8K each | 32K each | |||
| RTX 3060 12 GB | — | — | — | 11.6 GB |
| RTX 4060 Ti 16 GB | 6 | 1 | 55K | 15.4 GB |
| RTX 3090 24 GB FP8 without its speed-up here | 23 | 5 | all 125K | 23.4 GB |
| RTX 4090 24 GB | 23 | 5 | all 125K | 23.4 GB |
| RTX 5090 32 GB | 40 | 10 | all 125K | 31.0 GB |
| L40S 48 GB | 67 | 16 | all 125K | 44.0 GB |
| A100 80 GB FP8 without its speed-up here | 140 | 35 | all 125K | 78.2 GB |
| H100 80 GB | 130 | 32 | all 125K | 78.1 GB |
| RTX PRO 6000 Blackwell 96 GB | 164 | 41 | all 125K | 93.8 GB |
| DGX Spark (GB10) 128 GB unified | 193 | 48 | all 125K | 107 GB |
| H200 141 GB | 258 | 64 | all 125K | 138 GB |
| B200 180 GB | 337 | 84 | all 125K | 176 GB |
| 2× RTX 3060 12 GB tensor parallel · FP8 without its speed-up here | 19 | 4 | all 125K | 11.6 GB a card |
| 2× RTX 4060 Ti 16 GB tensor parallel | 35 | 8 | all 125K | 15.4 GB a card |
| Requests at once | 8K tokens each | 32K tokens each |
|---|---|---|
| 1 | 12.6 GB | 14.0 GB |
| 5 | 14.5 GB | 21.5 GB |
| 8 | 15.9 GB | 27.2 GB |
| 16 | 19.7 GB | 42.2 GB |
| 32 | 27.2 GB | 72.3 GB |
| 64 | 42.2 GB | 132 GB |
One card, with vLLM's small-card settings.
From the model card
Quantized to FP8 Version of olmOCR-7B-0825, using llmcompressor.
This is a release of the olmOCR model that's fine tuned from Qwen2.5-VL-7B-Instruct using the olmOCR-mix-0225 dataset.
Quick links:
The best way to use this model is via the olmOCR toolkit. The toolkit comes with an efficient inference setup via sglang that can handle millions of documents at scale.
This model expects as input a single document image, rendered such that the longest dimension is 1288 pixels.
The prompt must then contain the additional metadata from the document, and the easiest way to generate this is to use the methods provided by the olmOCR toolkit.
olmOCR is licensed under the Apache 2.0 license. olmOCR is intended for research and educational use. For more information, please see our Responsible Use Guidelines.
Quoted from the model card on Hugging Face. The full card is behind the Hugging Face link above.