tiron

For whole meetings (beyond a single 30s window), use the open-source harness at TrelisResearch/tiron, which adds chunking, cross-window speaker linking (ECAPA voice embeddings), and SRT/VTT/JSON output.

What's new (23 July 2026)

• Updated checkpoint with small performance gains (NOTSOFAR-1 37.55 → 36.23 pooled cpWER, AMI 35.24 → 34.68; the 21 July release's numbers are kept in the table for comparison).
• Expanded benchmarking: added MOSS-Transcribe-Diarize and AssemblyAI universal-3.5-pro under the identical scoring harness.
• Reproducibility: the exact evaluation meetings are published with attribution, and minimal replication scripts live in the harness repo (links below).

Benchmarks

Pooled corpus cpWER (lower is better) on held-out whole-meeting test sets, scored with identical references, normalization, and `` masking for every system:

Tiron leads both AssemblyAI models on every corpus (−19% macro vs universal-3-pro, −15% vs universal-3.5-pro). MOSS-Transcribe-Diarize (Apache-2.0, ~2.5B incl. encoder) decodes the whole meeting in a single 128k context — it leads on AMI and NOTSOFAR-1, and the two are effectively tied on ICSI. Google Gemini 3.1 Pro is competitive on short clips but truncates or fails on longer meetings and cannot be scored across full corpora.

MOSS numbers use a quality-preserving vLLM decode configuration. We found MOSS's scores are sensitive to serving configuration (an aggressive speed-oriented vLLM config cost it several points on NOTSOFAR-1), so we report its best-quality vLLM decode here.

Speed (× real-time, whole-meeting inference over all 17 meetings; median [range]). Each system's speed is measured in the same configuration as its accuracy numbers above:

Tiron is chunk-based, so a meeting's 30-second windows decode in parallel on one GPU — that is where its speed comes from. | AssemblyAI u3.5-pro | 34× [7×–94×] | cloud API round-trip (incl. upload/queue) |

Meetings evaluated (whole-meeting audio, far-field where applicable) — published with references and attribution as Trelis/tiron-eval-meetings, with minimal replication scripts in eval/ of the harness repo:

• AMI (CC BY 4.0, AMI consortium): ES2004a, IS1009a, TS3003a, EN2002a
• ICSI (CC BY 4.0, ICSI): Bmr013, Bmr018, Bro021
• NOTSOFAR-1 (CC BY 4.0, Microsoft): MTG_32040, MTG_32063, MTG_32072, MTG_32074, MTG_32092, MTG_32179, MTG_32185, MTG_32256, MTG_32257, MTG_32322

Scoring notes: cpWER is concatenated-permutation WER over whole meetings (transcription and speaker-attribution errors both count). Each corpus figure is pooled — total errors ÷ total reference words across that corpus's meetings — and the macro is the mean of the three corpus figures. On NOTSOFAR-1, stretches the human annotators marked `` (unintelligible) are masked from both hypothesis and reference for every system, so no system is rewarded for staying silent there. AMI and ICSI are unaffected by this mask.

Measurement uncertainty — whole-meeting cpWER on small corpora is a noisy instrument for every system, and all numbers here are single decoding runs. Individual meetings can move by a few points between runs (speaker-count estimation flips, and in single-context decoders one divergent token early in the decode can cascade across the meeting); serving configuration can shift some systems' corpus figures by several points (see the MOSS note above). Corpus figures should be read as ±1 point, and cross-system gaps under ~2 points as ties. Expect small differences when reproducing.

Output format

Per 30-second window the model emits speaker blocks with within-window timestamps:

Speaker indices are local to the window (first speaker to talk is ``). The harness links speakers across windows into stable meeting-level identities using ECAPA voice embeddings.

Usage with transformers (single window, ≤