Model reference · open weights
larger_clap_general is an open-weight embedding model from laion, listed in the AxForge catalogue. AxForge can bring it up on EU-owned hardware for you on request — with the licence handled where one is required.
About
Model TL;DR CLAP is to audio what CLIP is to image. This is an improved CLAP checkpoint, specifically trained on general audio, music and speech. Description CLAP (Contrastive Language-Audio Pretraining) is a neural network trained on a variety of (audio, text) pairs. It can be instructed in to predict the most relevant text snippet, given an audio, without directly optimizing for the task. The CLAP model uses a SWINTransformer to get audio features from a log-Mel spectrogram input, and a RoBERTa model to get text features. Both the text and audio features are then projected to a latent space with identical dimension. The dot product between the projected audio and text features is then used as a similar score. Usage You can use this model for zero shot audio classification or extracting audio and/or textual features. Uses Perform zero-shot audio classification Using pipeline Run the model: You can also get the audio and text embeddings using ClapModel Run the model on CPU: Run the model on GPU: Citation If you are using this model for your work, please consider citing the original paper:
Summarised from the published model card. Read the full card on the HuggingFace links below.
Specifications
| Maker | laion |
|---|---|
| Type | Embedding models |
| Variants | 1 |
| Runs with | transformers |
| Released | 2023-10-30 |
| Popularity | 316k downloads / month |
| Likes | 53 |
| Licence | Open weights |
How it works
Variants
Open weights ship in several sizes and precisions. One page, all the variants — pick the one that fits your GPU. VRAM figures are estimates from model size.
| Variant | Params | Precision | VRAM | Fits 16 GB | Weights |
|---|---|---|---|---|---|
| larger_clap_general | — | BF16 | — | — | Weights ↗ |
Using it via the API
Once AxForge deploys larger-clap-general for you, it answers on the OpenAI-compatible API — the same base URL and keys as every other model. (larger-clap-general below is illustrative; you get the exact model name on deployment.)
$ curl -sS https://api.axforge.ai/v1/embeddings \
-H "Authorization: Bearer $AXFORGE_API_KEY" \
-H "Content-Type: application/json" \
-d '{"model":"larger-clap-general","input":"text to embed"}'
Licence
Open weights under apache-2.0 — commercial use is permitted. Deploy it on AxForge EU hardware on request. Read the licence ↗
Explore