Model reference · open weights
trocr-large-handwritten-fr is an open-weight language model from agomberto. AxForge deploys and operates it for you on dedicated EU-owned hardware — with the licence handled where one is required.
Available as managed deployment — configured and operated for you on dedicated EU hardware, quoted per deployment.
What it is
| Released by | agomberto |
|---|---|
| Type | Language models |
| Task | Image→text |
| Runs with | transformers |
| Released | 2023-05-09 |
| Popularity | 1k downloads / month |
| Licence | Open weights |
About
TrOCR handwritten has not yet released for French, so we trained a French model for PoC purpose. Based on this model, it is recommended to collect more data to additionally train the 1st stage or perform fine-tuning as the 2nd stage.
It's a special case of the English large handwritten trOCR model introduced in the paper TrOCR: Transformer-based Optical Character Recognition with Pre-trained Models by Li et al. and first released in this repository as a TrOCR model fine-tuned on the IAM dataset. We worked with Marie Beigelman on this.
We decided to fine-tuned in two steps on two datasets and one generated dataset:
The TrOCR model is an encoder-decoder model, consisting of an image Transformer as encoder, and a text Transformer as decoder. The image encoder was initialized from the weights of BEiT, while the text decoder was initialized from the weights of RoBERTa.
Images are presented to the model as a sequence of fixed-size patches (resolution 16x16), which are linearly embedded. One also adds absolute position embeddings before feeding the sequence to the layers of the Transformer encoder. Next, the Transformer text decoder autoregressively generates tokens.
You can use the raw model for optical character recognition (OCR) on single text-line images.
We used heuristic parameters without separate hyperparameter tuning.
For the dev set we got those results
For the test set we got those results
Here is how to use this model in PyTorch:
from transformers import TrOCRProcessor, VisionEncoderDecoderModel, AutoTokenizer
from PIL import Image
import requests
url = "https://github.com/agombert/trocr-base-printed-fr/blob/main/sample_imgs/5.jpg"
response = requests.get(url)
img = Image.open(BytesIO(response.content))
processor = TrOCRProcessor.from_pretrained('microsoft/trocr-large-handwritten')
model = VisionEncoderDecoderModel.from_pretrained('agomberto/trocr-large-handwritten-fr')
tokenizer = AutoTokenizer.from_pretrained('agomberto/trocr-large-handwritten-fr')
pixel_values = (processor(images=image, return_tensors="pt").pixel_values)
generated_ids = model.generate(pixel_values)
generated_text = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]
@miscellaneous{
author = {Arnault Gombert & Marie Beigelman},
title = {TrOCR in French: adapt to french archives},
year = {2023}
}
From the published model card. Full card on the HuggingFace links in the sidebar.
Using it via the API
Once AxForge deploys trocr-large-handwritten-fr for you, it answers on the OpenAI-compatible API — the same base URL and keys as every other model. (trocr-large-handwritten-fr below is illustrative; you get the exact model name on deployment.)
$ curl -sS https://api.axforge.ai/v1/chat/completions \
-H "Authorization: Bearer $AXFORGE_API_KEY" \
-H "Content-Type: application/json" \
-d '{"model":"trocr-large-handwritten-fr","messages":[{"role":"user","content":"Hello"}]}'
Create an account — your API key is available in the console. 3M free tokens every 30 days with every new account.