embeddinggemma-medical

This is a sentence-transformers model finetuned from google/embeddinggemma-300m on the miriad/miriad-4.4M dataset (specifically the first 100.000 question-passage pairs from tomaarsen/miriad-4.4M-split). It maps sentences & documents to a 768-dimensional dense vector space and can be used for medical information retrieval, specifically designed for searching for passages (up to 1k tokens) of scientific medical papers using detailed medical questions.

This model has been trained using code from our EmbeddingGemma blogpost to showcase how the EmbeddingGemma model can be finetuned on specific domains/tasks for even stronger performance. It is not affiliated with Google.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: google/embeddinggemma-300m
• Maximum Sequence Length: 1024 tokens
• Output Dimensionality: 768 dimensions
• Similarity Function: Cosine Similarity
• Training Dataset:
  • miriad-4.4_m-split (the first 100.000 samples of the default subset)
• Language: en
• License: apache-2.0

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 1024, 'do_lower_case': False, 'architecture': 'Gemma3TextModel'})
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Dense({'in_features': 768, 'out_features': 3072, 'bias': False, 'activation_function': 'torch.nn.modules.linear.Identity'})
  (3): Dense({'in_features': 3072, 'out_features': 768, 'bias': False, 'activation_function': 'torch.nn.modules.linear.Identity'})
  (4): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("sentence-transformers/embeddinggemma-300m-medical")
# Run inference
queries = [
    "What are some potential limitations in projecting the future demand for joint replacement surgeries?\n",
]
documents = [
    "We also asked whether current trends are advancing according to earlier expectations [6] .\n\n Our study has several limitations. Our projections are based on the historical growth trajectory of joint replacement surgeries, and do not take into account potential limitations in the availability of surgeons or limited economic resources by private and public payers and hospitals in the future. For example, a shortage in the number of surgeons will have a substantial influence on the actual number of procedures that are performed. We also have not incorporated the potential for future alternative technologies, such as cartilage regeneration or tissue engineering, or drug therapies that limit the progression of joint diseases, which may preempt the need for TJR. We were also unable to account for the potential impact of changes in economy, which may place additional economic burden on patients to pay substantial out-of pocket expenses for these procedures, depending on their insurance coverage. Our study also did not consider potential changes in healthcare policies, such as adoption of volume standards or regionalization of TJR to high volume centers [5] , which could limit the access to care and decrease the future demand. The above economic, policy, and scientific factors cannot be readily incorporated in the statistical model. Our study was also focused on the procedural trends in the U.S.; followup research may include an analysis of trends in other countries, though the availability of historical TJR trends in other countries may be limited. Nonetheless, these limitations in no way diminish the importance of conducting and regularly updating surgical projections to help guide future research, surgeon training, and public health policy decisions. Our study also incorporated a more conservative projection, which relied only on the future changes in population growth, while maintaining current rates of adoption of TJR. Despite these limitations, our current findings are expected to have implications in the private coverage and reimbursement of joint replacement procedures in the future, as patients less than 65 years of age are not typically covered by Medicare, which today funds the majority of total joint replacement procedures in the United States.\n\n We found the relative size of the young patient population for TJR has grown between 1993 and 2006. While 25% to 32% of primary or revision TJRs were performed in patients less than 65 years old in 1993, these proportions have increased to 40% to 46% in the most recent NIS data. The increasing trend in younger patients undergoing TJR has also been reported for different, but partly overlapping, historical periods. For example, Jain et al. reported that the proportion of primary TKA patients aged less than 60 years increased from 12.5% to 19.5% (+56%) between 1990-1993 and 1998-2000 [4] . In addition, for patients aged under 70 years, the proportion increased by 9% fr