Florence-2-large

About

What Florence-2-large is

Florence-2: Advancing a Unified Representation for a Variety of Vision Tasks Model Summary This is a continued pretrained version of Florence-2-large model with 4k context length, only 0.1B samples are used for continue pretraining, thus it might not be trained well. In addition, OCR task has been updated with line separator ('\n'). COCO OD AP 39.8 This Hub repository contains a HuggingFace's transformers implementation of Florence-2 model from Microsoft. Florence-2 is an advanced vision foundation model that uses a prompt-based approach to handle a wide range of vision and vision-language tasks. Florence-2 can interpret simple text prompts to perform tasks like captioning, object detection, and segmentation. It leverages our FLD-5B dataset, containing 5.4 billion annotations across 126 million images, to master multi-task learning. The model's sequence-to-sequence architecture enables it to excel in both zero-shot and fine-tuned settings, proving to be a competitive vision foundation model. Resources and Technical Documentation: + Florence-2 technical report. + Jupyter Notebook for inference and visualization of Florence-2-large How to Get Started with the Model Use the code below to get started with the model. All models are trained with float16. Tasks This model is capable of performing different tasks through changing the prompts. First, let's define a function to run a prompt. Here are the tasks Florence-2 could perform: Caption Detailed Caption More Detailed Caption Caption to Phrase Grounding caption to phrase grounding task requires additional text input, i.e. caption. Caption to phrase grounding results format: {'\<CAPTIONTOPHRASEGROUNDING': {'bboxes': [[x1, y1, x2, y2], ...], 'labels': ['', '', ...]}} Object Detection OD results format: {'\<OD': {'bboxes': [[x1, y1, x2, y2], ...], 'labels': ['label1', 'label2', ...]} } Dense Region Caption Dense region caption results format: {'\<DENSEREGIONCAPTION' : {'bboxes': [[x1, y1, x2, y2], ...], 'labels': ['label1', 'label2', ...]} } Region proposal Dense region caption results format: {'\<REGIONPROPOSAL': {'bboxes': [[x1, y1, x2, y2], ...], 'labels': ['', '', ...]}} OCR OCR with Region OCR with region output f

Summarised from the published model card. Read the full card on the HuggingFace links below.

Specifications

What it is

How it works

How language models work

Variants

Sizes & precisions

Open weights ship in several sizes and precisions. One page, all the variants — pick the one that fits your GPU. VRAM figures are estimates from model size.

Using it via the API

Call it like any OpenAI endpoint

Once AxForge deploys florence-2-large for you, it answers on the OpenAI-compatible API — the same base URL and keys as every other model. (florence-2-large below is illustrative; you get the exact model name on deployment.)

$ curl -sS https://api.axforge.ai/v1/chat/completions \
  -H "Authorization: Bearer $AXFORGE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"florence-2-large","messages":[{"role":"user","content":"Hello"}]}'

Details

Languages, data & research

Papers

Licence

Open weights

Sources

Weights & code

Explore

More language models