dots-studio_dots3-note-prev

About

What dots-studio_dots3-note-prev is

Llamacpp imatrix Quantizations of dots3-note-prev by dots-studio Using <a href="https://github.com/ggml-org/llama.cpp/"llama.cpp</a release <a href="https://github.com/ggml-org/llama.cpp/releases/tag/b10569"b10569</a for quantization. Original model: https://huggingface.co/dots-studio/dots3-note-prev Model details: - Parameter count: 288B - Input support: text, image, audio (with mmproj file) - details - Speculative decoding: yes (MTP) - details - imatrix: yes - details How to run Prompt format Don't know which to choose? Grab Q4KM (171.11GB) - usually a good mix of size and performance. Download instructions available here Available files: Download a specific file: Downloading using the Hugging Face CLI First, make sure you have the Hugging Face CLI installed: Download a specific file: The files marked true in the Split column above are stored as multiple parts in a folder. To download all the parts to a local folder, run: You can either specify a new local-dir (dots-studiodots3-note-prev-Q80) or download them all in place (./) How to run These quants run with llama.cpp - installable in one line via llama.app: llama-server includes a built-in chat web UI, served at http://localhost:8080 by default. These quants were made with llama.cpp release b10569 - if this model's architecture is newly supported, you'll need that release or newer to run them. They also work in: LM Studio · koboldcpp · ramalama · Jan AI · Text Generation Web UI · LoLLMs · Atomic Chat Multimodal This model supports image and audio input. Alongside the quants, this repo includes the multimodal projector files mmproj-dots-studiodots3-note-prev-f16.gguf and mmproj-dots-studiodots3-note-prev-bf16.gguf, which pair with any quant above. llama.cpp downloads the mmproj automatically when using -hf as shown above; if you're loading files manually, pass it with --mmproj. MTP This model has MTP (Multi-Token Prediction) layers, and they are included in these quants MTP layers act as a built-in draft model, letting llama.cpp run speculative decoding for faster generation. To use them, add the following flag to your llama.cpp command: Note: the MTP layers are stored at Q40 in the imatrix quants (except for

Summarised from the published model card. Read the full card on the HuggingFace links below.

Specifications

What it is

How it works

How language models work

Variants

Sizes & precisions

Open weights ship in several sizes and precisions. One page, all the variants — pick the one that fits your GPU. VRAM figures are estimates from model size.

Using it via the API

Call it like any OpenAI endpoint

Once AxForge deploys dots-studio-dots3-note-prev for you, it answers on the OpenAI-compatible API — the same base URL and keys as every other model. (dots-studio-dots3-note-prev below is illustrative; you get the exact model name on deployment.)

$ curl -sS https://api.axforge.ai/v1/chat/completions \
  -H "Authorization: Bearer $AXFORGE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"dots-studio-dots3-note-prev","messages":[{"role":"user","content":"Hello"}]}'

Details

Languages, data & research

Licence

Open weights

Sources

Weights & code

Explore

More language models