Model reference · open weights
mgp-str is an open-weight language model from alibaba-damo, listed in the AxForge catalogue. AxForge can bring it up on EU-owned hardware for you on request — with the licence handled where one is required.
About
MGP-STR (base-sized model) MGP-STR base-sized model is trained on MJSynth and SynthText. It was introduced in the paper Multi-Granularity Prediction for Scene Text Recognition and first released in this repository. Model description MGP-STR is pure vision STR model, consisting of ViT and specially designed A^3 modules. The ViT module was initialized from the weights of DeiT-base, except the patch embedding model, due to the inconsistent input size. Images (32x128) are presented to the model as a sequence of fixed-size patches (resolution 4x4), which are linearly embedded. One also adds absolute position embeddings before feeding the sequence to the layers of the ViT module. Next, A^3 module selects a meaningful combination from the tokens of ViT output and integrates them into one output token corresponding to a specific character. Moreover, subword classification heads based on BPE A^3 module and WordPiece A^3 module are devised for subword predictions, so that the language information can be implicitly modeled. Finally, these multi-granularity predictions (character, subword and even word) are merged via a simple and effective fusion strategy. Intended uses & limitations You can use the raw model for optical character recognition (OCR) on text images. See the model hub to look for fine-tuned versions on a task that interests you. How to use Here is how to use this model in PyTorch: BibTeX entry and citation info
Summarised from the published model card. Read the full card on the HuggingFace links below.
Specifications
| Maker | alibaba-damo |
|---|---|
| Type | Language models |
| Parameters (lead) | 148M |
| Variants | 1 |
| Runs with | transformers |
| Released | 2022-11-23 |
| Popularity | 105k downloads / month |
| Likes | 65 |
| Licence | Commercial licence needed |
How it works
Variants
Open weights ship in several sizes and precisions. One page, all the variants — pick the one that fits your GPU. VRAM figures are estimates from model size.
| Variant | Params | Precision | VRAM | Fits 16 GB | Weights |
|---|---|---|---|---|---|
| mgp-str-base | 148M | BF16 | ~0.3 GB | ✓ | Weights ↗ |
Using it via the API
Once AxForge deploys mgp-str for you, it answers on the OpenAI-compatible API — the same base URL and keys as every other model. (mgp-str below is illustrative; you get the exact model name on deployment.)
$ curl -sS https://api.axforge.ai/v1/chat/completions \
-H "Authorization: Bearer $AXFORGE_API_KEY" \
-H "Content-Type: application/json" \
-d '{"model":"mgp-str","messages":[{"role":"user","content":"Hello"}]}'
Licence
The weights are open but its licence needs a commercial agreement for business use. AxForge can arrange that licence and host the model for you — you pay AxForge, we settle with the model’s maker. Ask us for a quote. Read the licence ↗