Model reference · open weights
BioLORD-2023 is an open-weight embedding model from FremyCompany, listed in the AxForge catalogue. AxForge can bring it up on EU-owned hardware for you on request — with the licence handled where one is required.
About
FremyCompany/BioLORD-2023 This model was trained using BioLORD, a new pre-training strategy for producing meaningful representations for clinical sentences and biomedical concepts. State-of-the-art methodologies operate by maximizing the similarity in representation of names referring to the same concept, and preventing collapse through contrastive learning. However, because biomedical names are not always self-explanatory, it sometimes results in non-semantic representations. BioLORD overcomes this issue by grounding its concept representations using definitions, as well as short descriptions derived from a multi-relational knowledge graph consisting of biomedical ontologies. Thanks to this grounding, our model produces more semantic concept representations that match more closely the hierarchical structure of ontologies. BioLORD-2023 establishes a new state of the art for text similarity on both clinical sentences (MedSTS) and biomedical concepts (EHR-Rel-B). This model is based on sentence-transformers/all-mpnet-base-v2 and was further finetuned on the BioLORD-Dataset and LLM-generated definitions from the Automatic Glossary of Clinical Terminology (AGCT). Sibling models This model is accompanied by other models in the BioLORD-2023 series, which you might want to check: - BioLORD-2023-M (multilingual model; distilled from BioLORD-2023) - BioLORD-2023 (best model after model averaging; this model) - BioLORD-2023-S (best hyperparameters; no model averaging) - BioLORD-2023-C (contrastive training only; for NEL tasks) You can also take a look at last year's model and paper: - BioLORD-2022 (also known as BioLORD-STAMB2-v1) Training strategy Summary of the 3 phases Contrastive phase: details Self-distallation phase: details Citation This model accompanies the BioLORD-2023: Learning Ontological Representations from Definitions paper. When you use this model, please cite the original paper as follows: Usage (Sentence-Transformers) This is a sentence-transformers model: It maps sentences & paragraphs to a 768 dimensional dense vector space and can be used for tasks like clustering or semantic search. This model has been finentuned for the biomedical domain. While it p
Summarised from the published model card. Read the full card on the HuggingFace links below.
Specifications
| Maker | FremyCompany |
|---|---|
| Type | Embedding models |
| Parameters (lead) | 109M |
| Context | 514 tokens |
| Variants | 1 |
| Runs with | sentence-transformers |
| Released | 2023-11-27 |
| Popularity | 429k downloads / month |
| Likes | 57 |
| Licence | Commercial licence needed |
How it works
Variants
Open weights ship in several sizes and precisions. One page, all the variants — pick the one that fits your GPU. VRAM figures are estimates from model size.
| Variant | Params | Precision | VRAM | Fits 16 GB | Weights |
|---|---|---|---|---|---|
| BioLORD-2023 | 109M | BF16 | ~0.3 GB | ✓ | Weights ↗ |
Using it via the API
Once AxForge deploys biolord-2023 for you, it answers on the OpenAI-compatible API — the same base URL and keys as every other model. (biolord-2023 below is illustrative; you get the exact model name on deployment.)
$ curl -sS https://api.axforge.ai/v1/embeddings \
-H "Authorization: Bearer $AXFORGE_API_KEY" \
-H "Content-Type: application/json" \
-d '{"model":"biolord-2023","input":"text to embed"}'
Licence
The weights are open but its licence needs a commercial agreement for business use. AxForge can arrange that licence and host the model for you — you pay AxForge, we settle with the model’s maker. Ask us for a quote. Read the licence ↗
Explore