aac-pictogram-retriever-ptbr

• ~157M parameters, runs comfortably on CPU (no GPU).
• One model, two jobs: retrieval (find the best pictogram among ~14k) and reranking (order a shortlist).
• Competitive with much larger cloud embedding APIs on this domain, while being open and self-hostable.
• Ships with a ready-to-serve GGUF build for llama.cpp: aac-pictogram-retriever-ptbr-GGUF (~190 MB RAM at Q8).

Model family & resources

What it's for

In AAC, a child or therapist searches for a word (e.g. "escovar os dentes", "quero água") and the app must surface the pictogram that best represents it, so the child can point to it to communicate. This is a short-query, closed-domain, Brazilian-Portuguese retrieval problem where general-purpose embedders and leaderboard rankings do not transfer well. This model is tuned specifically for it.

Intended use: semantic search and reranking of ARASAAC pictograms from pt-BR words/phrases, in AAC applications.

Not intended for: general-purpose sentence embeddings, languages other than Brazilian Portuguese, or domains outside pictogram/AAC search.

Usage

With sentence-transformers

The model uses asymmetric prompts — one for the query, one for the document (pictogram text). Always apply them.

from sentence_transformers import SentenceTransformer
import numpy as np

model = SentenceTransformer("tardellirs/aac-pictogram-retriever-ptbr", trust_remote_code=True)

QUERY_PROMPT = "task: search result | query: "
DOC_PROMPT   = "title: none | text: "

query = "escovar os dentes"
# Pictogram "documents": a short text per pictogram (labels + synonyms + keywords + ARASAAC tags)
docs = [
    "escovar os dentes; escovação. tags: hygiene, health, routine",
    "escova de dentes; escova. tags: hygiene, object",
    "pente; pentear. tags: hygiene, object",
]

q = model.encode(QUERY_PROMPT + query, normalize_embeddings=True)
d = model.encode([DOC_PROMPT + t for t in docs], normalize_embeddings=True)
scores = d @ q                      # cosine similarity
print(docs[int(np.argmax(scores))]) # -> the escovar-os-dentes pictogram

Pre-compute the document embeddings for your pictogram catalog once; at query time you only embed the query and take cosine similarities. The same cosine score is used both to retrieve (over the full catalog) and to rerank a candidate shortlist.

On CPU with llama.cpp (recommended for deployment)

Use the GGUF build — it serves the full pipeline (pooling + dense projections) and loads in ~190 MB at Q8:

# from tardellirs/aac-pictogram-retriever-ptbr-GGUF (keep tokenizer.model next to the .gguf)
llama-server -m aac-Q8_0.gguf --embedding --pooling mean --host 127.0.0.1 --port 8080
# POST {"input": "task: search result | query: escovar os dentes"} to /v1/embeddings ; then cosine + normalize

Quantization quality (cosine vs. the fp16 reference, near-lossless): Q8_0 0.9998, Q6_K 0.9991, Q4_K_M 0.9926.

Model details

• Architecture: SentenceTransformer — Transformer → mean Pooling → Dense (768→3072) → Dense (3072→768) → Normalize. Output dimension 768.
• Base model: tardellirs/colibri-embed-ptbr, a Brazilian-Portuguese model derived from Google's EmbeddingGemma-300M with its vocabulary trimmed to the most relevant pt-BR tokens (~157M parameters after trimming). Architecture and prompts are inherited from EmbeddingGemma.
• Matryoshka embeddings: trained with nested dimensions [768, 256, 128], so outputs can be truncated to 256 or 128 dims for cheaper storage/search with graceful quality loss.
• Prompts: query task: search result | query: , document title: none | text: .

Training

Training data is released as aac-pictogram-retrieval-ptbr-training. The model was adapted to the AAC pictogram domain by distilling relevance signal from a strong LLM judge over the ARASAAC catalog:

• Retrieval supervision: for each pictogram, an LLM generated varied pt-BR search terms a child, therapist or caregiver might type, giving broad coverage of the full catalog (query → pictogram pairs).
• Rerank supervision: for realistic communication-board contexts, an LLM assigned graded relevance scores to candidate pictograms per target word, providing both positives and hard negatives.
• Objective: contrastive retrieval loss (in-batch negatives) + a graded-distillation margin loss, wrap