Model reference · open weights
quora-distilroberta is an open-weight embedding model from cross-encoder. quora-distilroberta-base (FP32) weighs 164 MB; the smallest configuration that runs it is RTX 3060 12 GB.
Summary of the cross-encoder/quora-distilroberta-base model card, 2026-10-01
What it is
| Released by | cross-encoder |
|---|---|
| Released | 2022-03-02 |
| Parameters | 82M |
| VRAM | 164 MB for the weights |
What it runs on
| Card | Runs | Memory |
|---|---|---|
| RTX 3060 12 GB | fits | 11.6 GB |
| RTX 4060 Ti 16 GB | fits | 15.4 GB |
| RTX 3090 24 GB | fits | 23.4 GB |
| RTX 4090 24 GB | fits | 23.4 GB |
| RTX 5090 32 GB | fits | 31.0 GB |
| L40S 48 GB | fits | 44.0 GB |
| A100 80 GB | fits | 78.2 GB |
| H100 80 GB | fits | 78.1 GB |
| RTX PRO 6000 Blackwell 96 GB | fits | 93.8 GB |
| DGX Spark (GB10) 128 GB unified | fits | 107 GB |
| H200 141 GB | fits | 138 GB |
| B200 180 GB | fits | 176 GB |
From the model card
This model was trained using SentenceTransformers Cross-Encoder class.
This model was trained on the Quora Duplicate Questions dataset. The model will predict a score between 0 and 1 how likely the two given questions are duplicates.
Note: The model is not suitable to estimate the similarity of questions, e.g. the two questions "How to learn Java" and "How to learn Python" will result in a rather low score, as these are not duplicates.
Pre-trained models can be used like this:
from sentence_transformers import CrossEncoder
model = CrossEncoder('cross-encoder/quora-distilroberta-base')
scores = model.predict([('Question 1', 'Question 2'), ('Question 3', 'Question 4')])
You can use this model also without sentence_transformers and by just using Transformers AutoModel class
Quoted from the model card on Hugging Face. The full card is behind the Hugging Face link above.