Qwen3.8-GSQ-RCO

About

What Qwen3.8-GSQ-RCO is

GSQ-RCO GGUF release card, TEMPLATE (filled with Qwen3.8-27B as the example). To publish a new model, copy this folder and change only: 1. the YAML frontmatter above (basemodel, license) 2. every field marked [swap] (model name, filenames, one-line summaries) 3. the results: drop tools/results/<model.json, run python tools/makeplots.py tools/results/<model.json then paste the printed Markdown table into "Results". The header (banner + badges) is shared across all releases. NB: the YAML block must remain the very first bytes of the file (HF requirement). -- Qwen3.8-27B &middot; GSQ-RCO GGUFs Non-uniform GGUF quantizations produced with GSQ and RCO, with a vision projector for multimodal use. [](https://arxiv.org/abs/2604.18556) [](https://arxiv.org/abs/2605.00649) [](https://github.com/IST-DASLab/GSQ) [](https://github.com/IST-DASLab/RCO) [](https://github.com/IST-DASLab) [](#license) Overview This repository provides GGUF quantizations of Qwen3.8-27B at three sizes, together with the model's vision projector (mmproj) for multimodal use. In contrast to uniform quantization, which applies a single quantization type to all weight tensors, each model here assigns a separate quantization type to every tensor. The assignment is obtained by a gradient-based search that allocates precision according to per-tensor sensitivity, subject to a total size budget. The resulting files are standard GGUF and run unmodified in llama.cpp, Ollama, and LM Studio. Method summary. GSQ provides accurate low-bit scalar quantization of each tensor at a given quantization type; RCO assigns the per-tensor quantization types under a size budget. Together they yield a non-uniform GGUF at the requested size. Both methods were developed at the Deep Algorithms and Systems Lab (DASLab), Institute of Science and Technology Austria. Available files Files follow the convention <model-GSQ-RCO-<type.gguf, where the suffix names the quantization class; the table lists each file's true whole-file average bit-width. The mmproj file carries the vision encoder and projector at BF16; one copy serves all quantizations. The IQ3XXS model is the task-lossless operating point: it matches the base model on AIME25

Summarised from the published model card. Read the full card on the HuggingFace links below.

Specifications

What it is

How it works

How language models work

Variants

Sizes & precisions

Open weights ship in several sizes and precisions. One page, all the variants — pick the one that fits your GPU. VRAM figures are estimates from model size.

Using it via the API

Call it like any OpenAI endpoint

Once AxForge deploys qwen3-8-gsq-rco for you, it answers on the OpenAI-compatible API — the same base URL and keys as every other model. (qwen3-8-gsq-rco below is illustrative; you get the exact model name on deployment.)

$ curl -sS https://api.axforge.ai/v1/chat/completions \
  -H "Authorization: Bearer $AXFORGE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"qwen3-8-gsq-rco","messages":[{"role":"user","content":"Hello"}]}'

Details

Languages, data & research

Papers

Licence

Open weights

Sources

Weights & code

Explore

More language models