phikon

About

What phikon is

Model Card for Phikon-v2 Phikon-v2 is a Vision Transformer Large pre-trained with Dinov2 self-supervised method on PANCAN-XL, a dataset of 450M 20x magnification histology images sampled from 60K whole slide images. PANCAN-XL only incorporates publicly available datasets: CPTAC (6,193 WSI) and TCGA (29,502 WSI) for malignant tissue, and GTEx for normal tissue (13,302 WSI). Phikon-v2 improves upon Phikon, our previous foundation model pre-trained with iBOT on 40M histology images from TCGA (6k WSI), on a large variety of weakly-supervised tasks tailored for biomarker discovery. Phikon-v2 is evaluated on external cohorts to avoid any data contamination with PANCAN-XL pre-training dataset, and benchmarked against an exhaustive panel of representation learning and foundation models. Model Description - Developed by: Owkin, Inc - Model type: Pretrained vision backbone (ViT-L/16 via DINOv2) - Pretraining dataset: PANCAN-XL, sourced from public histology collections (TCGA, CPTAC, GTEx, TCIA and others). - Paper: Arxiv - License: Owkin non-commercical licence How To Use (Feature Extraction) The following code snippet allows you to extract features from histology images using Phikon-v2 (CLS token). These features can then be used for downstream applications such as ROI classification (via linear or knn probing), slide classification (via multiple instance learning), segmentation (via ViT-Adapter for instance), etc. Direct Use (with Pre-Extracted and Frozen Features) Phikon-v2 can be used with or without fine-tuning on different downstream applications, on top of which slide-classification using multiple instance learning algorithms (such as ABMIL). Downstream Use (Finetuning) You can fine-tune the model on tile-level downstream tasks. This Colab notebook allows you to fine-tune Phikon and Phikon-v2 using LoRa through the huggingface API. Training Details - Training data: PANCAN-XL, a pretraining dataset composed of 456,060,584 [224×224] histology images at 20× resolution, sampled from 60k H&E WSIs. - Training regime: fp16 using PyTorch-FSDP mixed-precision. - Training objective: DINOv2 SSL recipe with the following losses: - DINO self-distillation loss with multi-crop -

Summarised from the published model card. Read the full card on the HuggingFace links below.

Specifications

What it is

How it works

How embedding models work

Variants

Sizes & precisions

Open weights ship in several sizes and precisions. One page, all the variants — pick the one that fits your GPU. VRAM figures are estimates from model size.

Using it via the API

Call it like any OpenAI endpoint

Once AxForge deploys phikon for you, it answers on the OpenAI-compatible API — the same base URL and keys as every other model. (phikon below is illustrative; you get the exact model name on deployment.)

$ curl -sS https://api.axforge.ai/v1/embeddings \
  -H "Authorization: Bearer $AXFORGE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"phikon","input":"text to embed"}'

Details

Languages, data & research

Papers

Licence

Commercial licence needed

Sources

Weights & code

Explore

More embedding models