OmniRad-small

Key Features

• Radiology-focused foundation model pretrained on >1M radiological images
• Self-supervised learning based on a customized DINOv2 framework
• Task-agnostic encoder reusable across classification, segmentation, and multimodal pipelines
• Strong transferability across modalities (CT, MRI, X-ray, ultrasound)
• Radiomics-oriented design, emphasizing representation stability and reuse

Example Usage: Feature Extraction

from PIL import Image
from torchvision import transforms
import timm
import torch

# Load OmniRad-small from Hugging Face Hub
model = timm.create_model(
    "hf_hub:Snarcy/OmniRad-small",
    pretrained=True,
    num_classes=0  # return embeddings
)

model.eval()
device = "cuda" if torch.cuda.is_available() else "cpu"
model.to(device)

# Preprocessing
transform = transforms.Compose([
    transforms.Resize((224, 224)),
    transforms.ToTensor(),
    transforms.Normalize(
        mean=[0.485, 0.456, 0.406],
        std=[0.229, 0.224, 0.225],
    ),
])

# Load image
image = Image.open("path/to/radiology_image.png").convert("RGB")
x = transform(image).unsqueeze(0).to(device)

# Extract features
with torch.no_grad():
    embedding = model(x)  # shape: [1, 384]

Available Downstream Code

The official OmniRad repository provides end-to-end implementations for all evaluated downstream tasks:

👉 https://github.com/unica-visual-intelligence-lab/OmniRad

Including:

• Image-level classification (MedMNIST v2 benchmarks)
• Dense medical image segmentation (MedSegBench, frozen encoder + lightweight decoders)
• Radiological image captioning (BART-based vision–language framework)
• Full training, evaluation, and ablation scripts
• Reproducible experimental configurations matching the paper

Model Details

• Architecture: Vision Transformer (ViT-S)
• Patch size: 14
• Embedding dimension: 384
• Pretraining framework: Modified DINOv2 (global crops only)
• Pretraining dataset: RadImageNet (~1.2M radiological images)
• Input resolution: 224 × 224
• Backbone type: Encoder-only (no task-specific heads)

Pretraining Notes

• Local crops are removed to improve training stability and downstream transferability
• No feature collapse observed during training
• Same hyperparameter configuration used across small and base variants
• Designed to support frozen-backbone adaptation and lightweight fine-tuning

Intended Use

OmniRad is intended as a general-purpose radiological image encoder for:

• Image-level classification (e.g., disease or organ recognition)
• Dense prediction (e.g., medical image segmentation via adapters or decoders)
• Radiomics feature extraction
• Representation transfer across datasets, modalities, and institutions
• Exploratory vision–language research (e.g., radiological image captioning)

Not intended for direct clinical deployment without task-specific validation.

License

This project and the released model weights are licensed under the Creative Commons Attribution 4.0 International (CC BY 4.0) license.

Made with ❤️ by UNICA Visual Intelligence Lab