MiniCPM-V-4.6-Thinking-BNB

About

What MiniCPM-V-4.6-Thinking-BNB is

This repository hosts the bitsandbytes (NF4, 4-bit) quantized version of MiniCPM-V 4.6 Thinking. For the original BF16 weights and the full model card, please refer to openbmb/MiniCPM-V-4.6-Thinking. A Pocket-Sized MLLM for Ultra-Efficient Image and Video Understanding on Your Phone GitHub | MiniCPM Wiki(Chinese) | CookBook | Demo | Feishu (Lark) News [2026.05.17] ⭐️⭐️⭐️ We release the API service of MiniCPM-V 4.6, with a public free API key together! Try it now. MiniCPM-V 4.6 Thinking MiniCPM-V 4.6 Thinking is the long chain-of-thought reasoning variant of MiniCPM-V 4.6. It generates an explicit reasoning trace before producing the final answer, substantially boosting performance on complex multimodal reasoning, math, and OCR-heavy tasks, while keeping the same edge-friendly architecture (SigLIP2-400M vision encoder + Qwen3.5-0.8B LLM) and the mixed 4x/16x visual token compression of MiniCPM-V 4.6. Evaluation <!-- omit in toc -- Overall Performance (Thinking) High-Concurrency Throughput Single Request TTFT (ms) Examples <!-- omit in toc -- Overall MiniCPM-V 4.6 can be deployed across three mainstream end-side platforms — iOS, Android and HarmonyOS. The clips below are raw screen recordings on phone devices without edition. Usages Inference with Transformers <!-- omit in toc -- Installation <!-- omit in toc -- Note on CUDA compatibility: torchcodec (used for video decoding) may have compatibility issues with certain CUDA versions. For example, torch=2.11 bundles CUDA 13.1 by default, while environments with CUDA 12.x may encounter errors such as RuntimeError: Could not load libtorchcodec. Two workarounds: 1. Replace torchcodec with PyAV — supports both image and video inference without CUDA version constraints: bash pip install "transformers[torch]=5.7.0" torchvision av 2. Pin the CUDA version when installing torch to match your environment (e.g. CUDA 12.8): bash pip install "transformers=5.7.0" torchvision torchcodec --index-url https://download.pytorch.org/whl/cu128 Load Model <!-- omit in toc -- Image Inference <!-- omit in toc -- Video Inference <!-- omit in toc -- Advanced Parameters <!-- omit in toc -- You can customize image/video processing by passing ad

Summarised from the published model card. Read the full card on the HuggingFace links below.

Specifications

What it is

How it works

How language models work

Variants

Sizes & precisions

Open weights ship in several sizes and precisions. One page, all the variants — pick the one that fits your GPU. VRAM figures are estimates from model size.

Using it via the API

Call it like any OpenAI endpoint

Once AxForge deploys minicpm-v-4-6-thinking-bnb for you, it answers on the OpenAI-compatible API — the same base URL and keys as every other model. (minicpm-v-4-6-thinking-bnb below is illustrative; you get the exact model name on deployment.)

$ curl -sS https://api.axforge.ai/v1/chat/completions \
  -H "Authorization: Bearer $AXFORGE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"minicpm-v-4-6-thinking-bnb","messages":[{"role":"user","content":"Hello"}]}'

Details

Languages, data & research

Papers

Licence

Open weights

Sources

Weights & code

Explore

More language models