TheDrummer_Orion

About

What TheDrummer_Orion is

Llamacpp imatrix Quantizations of Orion-26B-A4B-v1 by TheDrummer Using <a href="https://github.com/ggml-org/llama.cpp/"llama.cpp</a release <a href="https://github.com/ggml-org/llama.cpp/releases/tag/b10630"b10630</a for quantization. Original model: https://huggingface.co/TheDrummer/Orion-26B-A4B-v1 Model details: - Parameter count: 26B - Input support: text, image and audio (with mmproj file) - details - Speculative decoding: no - imatrix: yes - details How to run Prompt format Don't know which to choose? Grab Q4KM (17.04GB) - usually a good mix of size and performance. Download instructions available here Available files: Download a specific file: Downloading using the Hugging Face CLI First, make sure you have the Hugging Face CLI installed: Download a specific file: How to run These quants run with llama.cpp - installable in one line via llama.app: llama-server includes a built-in chat web UI, served at http://localhost:8080 by default. These quants were made with llama.cpp release b10630 - if this model's architecture is newly supported, you'll need that release or newer to run them. They also work in: LM Studio · koboldcpp · ramalama · Jan AI · Text Generation Web UI · LoLLMs · Atomic Chat Multimodal This model supports image and audio input. Alongside the quants, this repo includes the multimodal projector files mmproj-TheDrummerOrion-26B-A4B-v1-f16.gguf and mmproj-TheDrummerOrion-26B-A4B-v1-bf16.gguf, which pair with any quant above. llama.cpp downloads the mmproj automatically when using -hf as shown above; if you're loading files manually, pass it with --mmproj. imatrix All quants made using imatrix option, with a calibration corpus rendered through this model's own chat template. The corpus pairs plain prose with tool-calling and reasoning conversations (corpus source data), encoded exactly as this model sees them at inference and processed with --parse-special, so chat-format special tokens contribute to the importance matrix. The corpus rendered for this model is included in this repo: TheDrummerOrion-26B-A4B-v1-calibration-v6.txt. The imatrix is available here: TheDrummerOrion-26B-A4B-v1-imatrix.gguf. Embed/output weights Some of these quants (Q3K

Summarised from the published model card. Read the full card on the HuggingFace links below.

Specifications

What it is

How it works

How language models work

Variants

Sizes & precisions

Open weights ship in several sizes and precisions. One page, all the variants — pick the one that fits your GPU. VRAM figures are estimates from model size.

Using it via the API

Call it like any OpenAI endpoint

Once AxForge deploys thedrummer-orion for you, it answers on the OpenAI-compatible API — the same base URL and keys as every other model. (thedrummer-orion below is illustrative; you get the exact model name on deployment.)

$ curl -sS https://api.axforge.ai/v1/chat/completions \
  -H "Authorization: Bearer $AXFORGE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"thedrummer-orion","messages":[{"role":"user","content":"Hello"}]}'

Details

Languages, data & research

Licence

Commercial licence needed

Sources

Weights & code

Explore

More language models