LFM2.5-VL-DSpark

LFM2.5-VL-3B-DSpark

LFM2.5-VL-3B-DSpark is an experimental speculative-decoding draft model that brings DSpark to vision-language models. It allows LiquidAI/LFM2.5-VL-3B to decode substantially faster without changing its output, for a minimal increase in memory footprint.

This is a drafter for LiquidAI/LFM2.5-VL-3B. In SGLang on a single H100, decoding runs up to 2.66× faster. On Apple silicon, it reaches up to 3.13× with MLX-VLM on an M5 Max and up to 2.14× with llama.cpp on an M3 Ultra.

Find more information about LFM2.5-VL-3B-DSpark in our blog post.

🗒️ Model Details

LFM2.5-VL-3B-DSpark is a DSpark speculative-decoding draft model with the following features:

• Target model: LiquidAI/LFM2.5-VL-3B
• Draft parameters: 279.5M (BF16)
• Backbone: 4 full attention layers, hidden_size=2048, intermediate_size=6144 with SiLU/SwiGLU, GQA with num_attention_heads=32 / num_key_value_heads=8, head_dim=64
• Extra heads: Markov head (rank 256) + confidence head
• Block size: 9 during training; 8 or 9 at inference, depending on hardware
• Vocabulary: 128,000

[!NOTE] On Apple silicon the drafter is run at block size 8 rather than 9.

Use each drafter checkpoint with its corresponding target model:

📊 Performance

Benchmarks

Speculative decoding is exact under greedy decoding: the target verifies every proposed token, so the generated text is what the target would have produced on its own. Under matched sampling settings at non-zero temperatures, speculative decoding preserves the target model's output distribution. You get the speedup, not a different model.

See LiquidAI/LFM2.5-VL-3B for quality benchmarks.

Draft acceptance

This table reports the mean number of draft tokens accepted per target verification pass at batch size 1 and temperature 0. Higher acceptance generally enables greater acceleration, but the measured speedup also depends on hardware and runtime overhead.

Following MMSpec, the evaluation covers General VQA, Text VQA, Image Captioning, Chart VQA, Complex Reasoning, and Multi-turn Conversation.

Measured inference speedup

This table reports the resulting runtime performance relative to the same target model without DSpark. Each cell is formatted as decode / end-to-end.

All measurements use 16-bit processing for both the vision encoder and language backbone. The H100 results use SGLang on one H100 80GB in BF16 at batch size 1, temperature 0, and block size 9. The Apple results use FP16 weights at batch size 1, temperature 0, block size 8, and up to 2,048 output tokens. Measurements were collected with Pipette.

🏃 Inference

LFM2.5-VL-3B-DSpark is supported by SGLang for NVIDIA GPUs and MLX-VLM for Apple silicon. For llama.cpp, use the GGUF checkpoint.

SGLang

Requires SGLang v0.5.19 or newer. Launch the target with the draft attached:

python -m sglang.launch_server \
  --model-path LiquidAI/LFM2.5-VL-3B \
  --speculative-algorithm DSPARK \
  --speculative-draft-model-path LiquidAI/L