Model reference · open weights
voiceclap-small is an open-weight embedding model from laion. AxForge deploys and operates it for you on dedicated EU-owned hardware — with the licence handled where one is required.
Available as managed deployment — configured and operated for you on dedicated EU hardware, quoted per deployment.
What it is
| Released by | laion |
|---|---|
| Type | Embedding models |
| Task | Embeddings |
| Parameters (lead) | 113M |
| Runs with | transformers |
| Based on | laion/voiceclap-small |
| Released | 2026-06-11 |
| Popularity | 2k downloads / month |
| Licence | Commercial licence needed |
About
Voice-text contrastive (CLAP-style) embedding model — the successor to
laion/voiceclap-small,
trained with emotion-led MOSS-Audio short captions. Better than v1 on
every benchmark we measure, at identical size and inference cost.
Same dual-tower architecture as v1: a
BUD-E-Whisper_V1.1 audio
encoder paired with
sentence-transformers/all-MiniLM-L6-v2
on the text side, joined by an MLP projection on each side and trained with
the SigLIP sigmoid contrastive loss.
| Architecture | dual-tower CLAP (BUD-E-Whisper-Small + MiniLM-L6-v2) |
| Audio encoder | Whisper-style: 12 layers × 768 dim × 12 heads, 80-mel input @ 16 kHz |
| Text encoder | BERT/MiniLM, 6 layers × 384 dim, mean-pooled |
| Joint embedding | 768-d, L2-normalised |
| Loss | SigLIP (sigmoid contrastive) |
| Total parameters | ~110 M |
| Training | 40 M samples (20 epochs × 2 M), best checkpoint epoch 19 |
v1 sampled k=2 uniformly-chosen MOSS-Audio attribute sentences per clip
as the caption. v2 replaces this with an emotion-led short caption: the
MOSS-Audio-8B-Thinking EMO sentence (a direct natural-language description
of the emotional state) plus one randomly sampled talking-style sentence,
re-drawn every epoch. Captions stay 50/50 blended with each corpus's original
captions. The emotion-first structure concentrates contrastive signal on the
emotion subspace without sacrificing style coverage.
| Benchmark | v1 (released) | v2 (this model) | Δ |
|---|---|---|---|
| EmoNet-Voice top-1 | 0.0902 | 0.1015 | +13% rel |
| EmoNet-Voice Spearman ρ | 0.2280 | 0.2561 | +12% rel |
| MAEB-voice mean (8 tasks) | 0.3861 | 0.3893 | +0.8% |
The ρ gain also clears every arm of the v1 caption-sampling sweep (best: 0.2399 at k=2). MAEB-voice shows no general-speech regression.
Trained on the open 9-corpus mixture used in the VoiceNet paper:
emolia-balanced-5M-subset (annotated subset of Emilia)laions_got_talent_clean_with_captionsmajestrino-datasynthetic_vocal_bursts + improved_synthetic_vocal_burstsears, expresso, voxceleb1, voxceleb2 (FCaps captions)MOSS-Audio-8B-Thinking annotations (18 prompt groups, 61 attribute values per clip) provide the EMO + style sentences for the three large corpora.
import torch
from transformers import AutoModel, AutoTokenizer
model = AutoModel.from_pretrained("laion/voiceclap-small-v2", trust_remote_code=True).eval()
tok = AutoTokenizer.from_pretrained("laion/voiceclap-small-v2")
# audio: raw mono waveform at 16 kHz
import soundfile as sf
wav, sr = sf.read("clip.wav", dtype="float32")
audio_emb = model.encode_waveform(torch.from_numpy(wav))
# text
t = tok(["a person speaking with quiet pride in their voice"], padding=True, return_tensors="pt")
text_emb = model.encode_text(t["input_ids"], attention_mask=t["attention_mask"])
score = (audio_emb @ text_emb.T).item()
Conversion from the training checkpoint was verified functionally against the original open_clip implementation (cosine ≥ 0.9999 on both towers).
laion/voiceclap-large-v2 — 7B single-tower successor trained with Prototypical Contrastive losslaion/voiceclap-small, laion/voiceclap-large — v1 releasescc-by-nc-4.0
From the published model card. Full card on the HuggingFace links in the sidebar.
Using it via the API
Once AxForge deploys voiceclap-small for you, it answers on the OpenAI-compatible API — the same base URL and keys as every other model. (voiceclap-small below is illustrative; you get the exact model name on deployment.)
$ curl -sS https://api.axforge.ai/v1/embeddings \
-H "Authorization: Bearer $AXFORGE_API_KEY" \
-H "Content-Type: application/json" \
-d '{"model":"voiceclap-small","input":"text to embed"}'
Create an account — your API key is available in the console. 3M free tokens every 30 days with every new account.