Model reference · open weights
Sa2VA-Qwen3-VL-SAM3 is an open-weight language model from ByteDance, listed in the AxForge catalogue. AxForge can bring it up on EU-owned hardware for you on request — with the licence handled where one is required.
About
Sa2VA-Qwen3-VL-4B-SAM3 Sa2VA-Qwen3-VL-4B-SAM3 is a Sa2VA model built on Qwen3-VL-4B-Instruct with a SAM3 PVS-tracker grounding encoder. The MLLM predicts a [SEG] token whose hidden state conditions the SAM3 mask decoder, producing dense image and video referring segmentation alongside open-ended chat. This checkpoint is self-contained: the SAM3 grounding code is vendored into the repository, so it loads with trustremotecode=True without any extra packages. Results Image referring segmentation (cIoU): Video referring segmentation (J&F): Grounded conversation generation (GCG, val): Usage Load in bfloat16 (the lmhead is kept in higher precision; torchdtype="auto" mixes dtypes and fails). For video, pass video=[frame0, frame1, ...] (a list of PIL images) instead of image. Notes - SAM3 grounding input resolution is 1008. - Built on Sa2VA. Citation If you find this project useful in your research, please consider citing:
Summarised from the published model card. Read the full card on the HuggingFace links below.
Specifications
| Maker | ByteDance |
|---|---|
| Type | Language models |
| Parameters (lead) | 5.3B |
| Variants | 1 |
| Runs with | transformers |
| Released | 2026-06-10 |
| Popularity | 395 downloads / month |
| Likes | 2 |
| Licence | Open weights |
How it works
Variants
Open weights ship in several sizes and precisions. One page, all the variants — pick the one that fits your GPU. VRAM figures are estimates from model size.
| Variant | Params | Precision | VRAM | Fits 16 GB | Weights |
|---|---|---|---|---|---|
| Sa2VA-Qwen3-VL-4B-SAM3 | 5.3B | BF16 | ~12.2 GB | ✓ | Weights ↗ |
Using it via the API
Once AxForge deploys sa2va-qwen3-vl-sam3 for you, it answers on the OpenAI-compatible API — the same base URL and keys as every other model. (sa2va-qwen3-vl-sam3 below is illustrative; you get the exact model name on deployment.)
$ curl -sS https://api.axforge.ai/v1/chat/completions \
-H "Authorization: Bearer $AXFORGE_API_KEY" \
-H "Content-Type: application/json" \
-d '{"model":"sa2va-qwen3-vl-sam3","messages":[{"role":"user","content":"Hello"}]}'
Details
Tags
Licence
Open weights under apache-2.0 — commercial use is permitted. Deploy it on AxForge EU hardware on request. Read the licence ↗