Model reference · open weights
kosmos-2-patch14-224 is an open-weight language model from microsoft, listed in the AxForge catalogue. AxForge can bring it up on EU-owned hardware for you on request — with the licence handled where one is required.
About
Kosmos-2: Grounding Multimodal Large Language Models to the World This Hub repository contains a HuggingFace's transformers implementation of the original Kosmos-2 model from Microsoft. How to Get Started with the Model Use the code below to get started with the model. Tasks This model is capable of performing different tasks through changing the prompts. First, let's define a function to run a prompt. Here are the tasks Kosmos-2 could perform: Multimodal Grounding • Phrase Grounding • Referring Expression Comprehension Multimodal Referring • Referring expression generation Perception-Language Tasks • Grounded VQA • Grounded VQA with multimodal referring via bounding boxes Grounded Image captioning • Brief • Detailed Draw the bounding bboxes of the entities on the image Once you have the entities, you can use the following helper function to draw their bounding bboxes on the image: Here is the annotated image: BibTex and citation info
Summarised from the published model card. Read the full card on the HuggingFace links below.
Specifications
| Maker | microsoft |
|---|---|
| Type | Language models |
| Parameters (lead) | 1.7B |
| Variants | 1 |
| Runs with | transformers |
| Released | 2023-10-02 |
| Popularity | 189k downloads / month |
| Likes | 183 |
| Licence | Open weights |
How it works
Variants
Open weights ship in several sizes and precisions. One page, all the variants — pick the one that fits your GPU. VRAM figures are estimates from model size.
| Variant | Params | Precision | VRAM | Fits 16 GB | Weights |
|---|---|---|---|---|---|
| kosmos-2-patch14-224 | 1.7B | BF16 | ~3.8 GB | ✓ | Weights ↗ |
Using it via the API
Once AxForge deploys kosmos-2-patch14-224 for you, it answers on the OpenAI-compatible API — the same base URL and keys as every other model. (kosmos-2-patch14-224 below is illustrative; you get the exact model name on deployment.)
$ curl -sS https://api.axforge.ai/v1/chat/completions \
-H "Authorization: Bearer $AXFORGE_API_KEY" \
-H "Content-Type: application/json" \
-d '{"model":"kosmos-2-patch14-224","messages":[{"role":"user","content":"Hello"}]}'
Details
Tags
Licence
Open weights under mit — commercial use is permitted. Deploy it on AxForge EU hardware on request. Read the licence ↗