UniverSat

Highlights

• 🌐 Universal. One weight set processes many modality combinations and arbitrary resolutions — optical, SAR, hyperspectral, and elevation — without channel filtering or resampling.
• 📏 Resolution-flexible. The output spatial resolution is chosen at inference and decoupled from the input patch size: coarse maps, native resolution, or per-pixel features from the same forward pass.
• 🔍 Granular. A sub-patch skip cross-attention recovers fine spatial detail (field boundaries, roads) beyond patch-level embeddings.
• 🧊 Frozen-backbone friendly. Competitive with ~9K-parameter linear probes — strong in low-label regimes.

Usage

The model is published with PyTorchModelHubMixin, so from_pretrained pulls the weights (config.json + model.safetensors) straight from this repo:

from hubconf import UniverSat   # from a local checkout on your path

model = UniverSat.from_pretrained("g-astruc/UniverSat").eval()

Equivalently, through Torch Hub — same weights, same tracked download, no local checkout needed:

import torch

model = torch.hub.load("gastruc/UniverSat", "from_pretrained").eval()

Loading requires huggingface_hub (and safetensors); building the model needs only torch.

Encode any combination of sensors

model.encode(...) looks up per-modality wavelengths, physical resolution, and sub-patch factors automatically from a built-in registry, so you only pass {modality_name: tensor}:

# Snapshot modalities: (B, C, H, W). Time series: (B, T, C, H, W) + a "_dates" tensor.
data = {
    "spot":     torch.randn(2,  3, 360, 360),       # 1 m VHR RGB snapshot
    "s2":       torch.randn(2, 20, 10,  36,  36),   # 10 m Sentinel-2 time series
    "s2_dates": torch.randint(0, 365, (2, 20)),     # day-of-year per timestamp
    "s1":       torch.randn(2, 12,  3,  36,  36),   # 10 m Sentinel-1 (VV, VH, ratio)
    "s1_dates": torch.randint(0, 365, (2, 12)),
    "dsm":      torch.randn(2,  1,  12,  12),       # 30 m elevation snapshot
}

features, _ = model.encode(data, patch_size=40, output_grid=36)
# features: (2, 1296, 768)  ->  a 36×36 dense feature grid (register tokens stripped for you)

• patch_size — patch size in metres (patch_size=40 → 40 m patches; scale = patch_size / 10 internally).
• output_grid — side G of the output grid (a G×G map, G² tokens), decoupled from the input patch size. The same model + inputs produce coarse or per-pixel maps just by changing it:

patch, _   = model.encode(data, patch_size=40, output_grid=9)     #   9×9   patch-level
dense, _   = model.encode(data, patch_size=40, output_grid=36)    #  36×36  dense
highres, _ = model.encode(data, patch_size=40, output_grid=180)   # 180×180 high-res

Unseen sensors? Pass the sensor's wavelengths={...} (optical/hyperspectral), polarization codes (SAR), input_res={...}, and subpatches={...} overrides to encode(...). The UPE uses these as positional encodings — no retraining needed.

Inputs should be normalised (per-channel z-score). For the low-level forward(...) API (explicit wavelengths, latent grid, masking), see hubconf.py.

Supported sensors

The encoder accepts any combination of the registered modalities below (and more — see modality_registry.py). Time-series modalities take a 5-D tensor plus a "_dates" companion (day-of-year, Jan 1 = 0).

Training

UniverSat is pre-trained self-supervised on 13 sensors from 7 datasets with a combination of latent multimodal masked modeling (LM³) and cross-modal contrastive learning under aggressive (~90%) masking across channels, time, space, and modalities.

Combined coverage: spatial resolution 0.1–300 m, temporal depth 1–150 images/year, spectral width 1–396 channels. Fold 1 of PASTIS