Qwen3-VL

About

What Qwen3-VL is

Qwen3VL-8B-Instruct-NVFP4 This is an NVFP4 quantized version of Qwen3-VL-8B-Instruct, a powerful vision-language model for multimodal understanding and generation tasks. Model Details Base Model - Base Model: Qwen/Qwen3-VL-8B-Instruct - Architecture: Qwen3VLForConditionalGeneration - Model Type: Vision-Language Model (VLM) Quantization Details - Quantization Method: NVFP4 with SmoothQuant - Quantization Tool: llmcompressor - Smoothing Strength: 0.8 - Calibration Dataset: lmms-lab/flickr30k (512 samples from test split) - Max Sequence Length: 32,768 tokens Quantization Configuration - Weight Quantization: NVFP4 (4-bit NVIDIA Floating Point) - Strategy: Channel-wise - Observer: MinMax - Symmetric: True - Activation Quantization: NVFP4 - Strategy: Token-wise - Dynamic scaling: Enabled - Symmetric: True Excluded Modules The following modules were excluded from quantization to maintain model quality: - lmhead (language model head) - Visual encoder modules (model.visual.) - MLP gate projections (.mlp.gate$) Model Use Installation Basic Usage Using with vLLM For faster inference, you can use this model with vLLM: Performance Memory Benefits - Reduced Memory Footprint: NVFP4 quantization significantly reduces model size and memory requirements (approximately 80% reduction compared to BF16) - Faster Inference: Lower precision enables faster computation on modern GPUs with NVFP4 support (2.5-3.5x speedup) - Optimized for NVIDIA Hardware: NVFP4 is optimized for NVIDIA GPUs, especially Jetson Thor platform Quality This quantized model maintains high quality for vision-language tasks while significantly reducing memory usage. The SmoothQuant technique helps preserve model accuracy during quantization. Typical quality degradation is 2-5% compared to the full-precision model. Training Details Quantization Process 1. Calibration: Used 512 samples from the flickr30k test dataset 2. SmoothQuant: Applied with smoothing strength of 0.8 to improve quantization quality 3. Sequential Processing: Applied quantization sequentially to Qwen3VLTextDecoderLayer modules Hardware - Quantization was performed on NVIDIA GPUs with CUDA support - Optimized for NVIDIA Jetson Thor platform Limitati

Summarised from the published model card. Read the full card on the HuggingFace links below.

Specifications

What it is

How it works

How language models work

Variants

Sizes & precisions

Open weights ship in several sizes and precisions. One page, all the variants — pick the one that fits your GPU. VRAM figures are estimates from model size.

Using it via the API

Call it like any OpenAI endpoint

Once AxForge deploys jeildlwlrma-qwen3-vl for you, it answers on the OpenAI-compatible API — the same base URL and keys as every other model. (jeildlwlrma-qwen3-vl below is illustrative; you get the exact model name on deployment.)

$ curl -sS https://api.axforge.ai/v1/chat/completions \
  -H "Authorization: Bearer $AXFORGE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"jeildlwlrma-qwen3-vl","messages":[{"role":"user","content":"Hello"}]}'

Details

Languages, data & research

Licence

Open weights

Sources

Weights & code

Explore

More language models