AtmicEmbedding

Benchmarks

Three-way comparison — base intfloat/multilingual-e5-large (zero domain fine-tuning) vs. v2 vs. v3 — on two test sets.

Own held-out test data (275 rows, 581-passage corpus):

v3 beats both base and v2 by a wide margin here — the larger, more diverse training set (theoretical dialogue data plus paraphrase/scenario questions) generalizes well beyond either the un-fine-tuned base model or v2's narrower specialization.

Full 161_pass dataset (644 triplets — v2's OWN original training data, used as a retention / catastrophic-forgetting check):

Honest tradeoff, stated plainly: v3's retention on 161_pass is the weakest of any AtmicEmbedding release — it has round-tripped to roughly base-model performance on this specific corpus (R@1 0.571 vs base's 0.592), losing most of the specialization v1→v2 training added there. This is a real cost of training on ~1.75x more data (3,176 vs the prior round's 1,808 rows) at the same epoch/learning-rate/freeze-layer budget, which pulls the model further from v2's weights. Notably, v2 itself underperforms base on the newer, more diverse own-test-data set (0.532 vs 0.611 accuracy) — v2 appears overfit to 161_pass's narrower style, and v3 trades that narrow strength for broader generalization. If 161_pass-specific retrieval quality matters for your use case, evaluate v3 there directly before switching from v2.