Trinity-Nano-Block

About

What Trinity-Nano-Block is

src="https://cdn-uploads.huggingface.co/production/uploads/6435718aaaef013d1aec3b8b/i-v1KyAMOWmgVGeic9WJ.png" alt="Arcee Trinity Nano Preview" style="max-width: 100%; height: auto;" Trinity Nano Preview FP8-Block Trinity Nano Preview is a preview of Arcee AI's 6B MoE model with 1B active parameters. It is the small-sized model in our new Trinity family, a series of open-weight models for enterprise and tinkerers alike. This is a chat tuned model, with a delightful personality and charm we think users will love. We note that this model is pushing the limits of sparsity in small language models with only 800M non-embedding parameters active per token, and as such may be unstable in certain use cases, especially in this preview. This is an experimental release, it's fun to talk to but will not be hosted anywhere, so download it and try it out yourself! Trinity Nano Preview is trained on 10T tokens gathered and curated through a key partnership with Datology, building upon the excellent dataset we used on AFM-4.5B with additional math and code. Training was performed on a cluster of 512 H200 GPUs powered by Prime Intellect using HSDP parallelism. More details, including key architecture decisions, can be found on our blog here This repository contains the FP8 block-quantized weights of Trinity-Nano-Preview (FP8 weights and activations with per-block scaling). Model Details Model Architecture: AfmoeForCausalLM Parameters: 6B, 1B active Experts: 128 total, 8 active, 1 shared Context length: 128k Training Tokens: 10T License: OpenMDW-1.1 Quantization Details - Scheme: FP8 Block (FP8 weights and activations, per-block scaling with E8M0 scale format) - Format: compressed-tensors - Intended use: High-throughput FP8 deployment of Trinity-Nano-Preview with near-lossless quality, optimized for NVIDIA Hopper/Blackwell GPUs - Supported backends: DeepGEMM, vLLM CUTLASS, Triton Running our model - VLLM - Transformers VLLM Supported in VLLM release 0.18.0+ with DeepGEMM FP8 MoE acceleration. Serving the model with DeepGEMM enabled: Serving without DeepGEMM (falls back to CUTLASS/Triton): Transformers Use the main transformers branch License Trinity-Nano-Preview-FP8-Block is relea

Summarised from the published model card. Read the full card on the HuggingFace links below.

Specifications

What it is

How it works

How language models work

Variants

Sizes & precisions

Open weights ship in several sizes and precisions. One page, all the variants — pick the one that fits your GPU. VRAM figures are estimates from model size.

Using it via the API

Call it like any OpenAI endpoint

Once AxForge deploys trinity-nano-block for you, it answers on the OpenAI-compatible API — the same base URL and keys as every other model. (trinity-nano-block below is illustrative; you get the exact model name on deployment.)

$ curl -sS https://api.axforge.ai/v1/chat/completions \
  -H "Authorization: Bearer $AXFORGE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"trinity-nano-block","messages":[{"role":"user","content":"Hello"}]}'

Details

Languages, data & research

Licence

Commercial licence needed

Sources

Weights & code

Explore

More language models