SciCore-Omics

A tri-modal foundation model unifying histology, spatial transcriptomics, and biological language

🔍 Overview

SciCore-Omics is a tri-modal biomedical foundation model that connects histology images, spatial transcriptomic profiles, and biological language for spatial biology and pathology-related reasoning.

The model introduces a gene-aware branch based on NicheFormer + Gene Q-Former + Gene Projector, enabling transcriptomic information to be aligned with the language-model token space.

SciCore-Omics supports:

• 🖼️ image-only reasoning;
• 🧬 gene-only reasoning;
• 🖼️🧬 joint image-gene reasoning;
• 💬 natural-language biomedical interpretation.

✨ Highlights

• Tri-modal modeling of histology, spatial transcriptomics, and language
• Gene-aware transcriptomic encoding with NicheFormer
• Unified image-gene-text reasoning in the language-model space
• Designed for spatial biology, pathology reasoning, and biomedical interpretation
• Open-source model weights, code, and demo

🚀 Quick Start

This Hugging Face repository hosts the model weights.

For full inference and training code, please refer to the GitHub repository:

git clone https://github.com/OpenBMB/Scicore-Omics.git
cd Scicore-Omics

Download the model weights:

huggingface-cli download openbmb/SciCore-Omics \
  --local-dir ./weights/SciCore-Omics

Minimal loading example:

import torch
from transformers import AutoModel, AutoTokenizer, AutoProcessor

model_path = "openbmb/SciCore-Omics"

processor = AutoProcessor.from_pretrained(
    model_path,
    trust_remote_code=True
)

tokenizer = AutoTokenizer.from_pretrained(
    model_path,
    trust_remote_code=True
)

model = AutoModel.from_pretrained(
    model_path,
    trust_remote_code=True,
    torch_dtype=torch.bfloat16,
    device_map="auto"
)

model.eval()

For complete examples, please see:

https://github.com/OpenBMB/Scicore-Omics/tree/main/eval

📦 Resources

⚠️ Limitations

SciCore-Omics is released for research use only.

It may generate inaccurate or incomplete biomedical interpretations and should not be used as a standalone clinical diagnostic or treatment recommendation system.

📚 Citation

@misc{xiao2026scicoreomics,
  title  = {SciCore-Omics: a tri-modal foundation model unifying histology, spatial transcriptomics and language for spatial biology},
  author = {Xiao, Xinyu and Li, Yunfei and Zeng, Zheni and others},
  year   = {2026},
  note   = {Manuscript in preparation}
}

📄 License

This project is released under the Apache-2.0 License.