Muse-Glimmer-ExecuTorch-PTE

Why an export, rather than a per-backend port. Local runtimes usually reimplement a model by hand for each target they support. That holds up for a plain text transformer; it does not hold up for a novel architecture with multimodal input and block-diffusion speculative decoding, where every backend would need its own rewrite of all three. ExecuTorch inverts that: the model and its decoding strategy are written once in PyTorch, and torch.export lowers the whole graph ahead of time — Triton on CUDA, MLX-native and custom Metal on Apple Silicon. The files in this repo are the output of that export. The PyTorch team's announcement makes the full argument.

⚠️ Read this before you download

This repo is 372 GB in total. A bare hf download meta-models/Muse-Glimmer-30B-ExecuTorch-PTE will try to pull all 16 variants. Always use --include to fetch exactly one variant (17.9–31.5 GB) plus the shared root files. The commands below do this.

On CUDA, the .pte is not the model. For every sm80+ptx variant the weights live in the .ptd (19–31 GB) and the .pte is only 16–35 MB. Both files are required. If you grabbed a 25 MB .pte and nothing works, this is why.

What's in this repo

16 variant directories, named by a fixed scheme:

muse-glimmer--128K---

What each directory contains

Every directory follows the same three rules — there are no exceptions:

1. Always: .pte
2. If and only if sm80+ptx: .ptd — the CUDA delegate blob holding the weights
3. If and only if text-image: pos_embed.bin — precomputed image position embeddings

A dflash directory is not an exception to rule 1. Target and drafter are exported together into that single .pte and share their token embeddings and output head, so the drafter costs far less than a second model would.

The artifacts are named after their own directory.

The artifacts are named after their own directory. They are not model.pte / aoti_cuda_blob.ptd — those are the filenames a local export_solo run writes, and they do not exist anywhere in this repo. Copy-pasting a quickstart that references them will fail. Use the $VARIANT pattern below and you cannot get this wrong.

Shared files at the repo root

tokenizer.json, tokenizer_config.json, chat_template.jinja, LICENSE, USAGE_POLICY.md. You need the three tokenizer/template files alongside whichever variant you pick — the server's --hf-tokenizer loads that directory to render prompts and tool definitions.

Download sizes (per directory, .pte + .ptd + pos_embed.bin)

Pick a variant

1. Backend — your hardware decides. Apple Silicon → metal. NVIDIA SM80+ → sm80+ptx. There is no CPU variant.
2. Quant — k-quant-17G for 24 GB of VRAM/unified memory, k-quant-dynamic for 32 GB. dynamic is measurably closer to full precision (see Fitting the Model on Your Device).
3. Modality — take text-image only if you actually send images; it costs memory and download size.
4. Decoding — dflash is meaningfully faster on capable GPUs but adds the drafter's memory and download cost. solo is the smaller, simpler starting point.

If you're unsure, start with muse-glimmer-k-quant-17G-128K-text-solo-.

Download one variant

Set the variant once; every command below derives its paths from it.

REPO=meta-models/Muse-Glimmer-30B-ExecuTorch-PTE
VARIANT=muse-glimmer-k-quant-17G-128K-text-solo-sm80+ptx   # ← change this
LOCAL_DIR=./muse-glimmer-pte

hf download "$REPO" \
  --include "$VARIANT/*" \
  --include "tokenizer.json" \
  --include "tokenizer_config.json"