aloe-dino-in1k-lp

Why ALOEv2 (qualitative difference vs. previous models)

Fig 2a — dense correspondence & surface normals across model sizes. ALOEv2 (orange) tracks the DINOv3 teacher closely on NAVI / ScanNet / SPair / surface-normals, while the original ALOE (grey, dotted) lags far behind.

NYUv2 depth probe. δ<1.25 (higher better) and RMSE (lower better) by size:

Fig 3 — discriminative quality (ImageNet-1k kNN@20 + Linear Probe).

GridPG localization. ALOEv2 inherent B-cos localization (higher is better):

ImageNet-1k (base): kNN@20 81.43, Linear Probe 83.92.

Usage — backbone (feature extraction)

from transformers import AutoImageProcessor, AutoModel

repo_id = "rmaser/aloe-v2-dinov3-base"
processor = AutoImageProcessor.from_pretrained(repo_id, trust_remote_code=True)
model = AutoModel.from_pretrained(repo_id, trust_remote_code=True)
model.eval()

Hidden states are pre-norm

With output_hidden_states=True, hidden_states[i] is the raw output of block i — in particular hidden_states[-1] is not last_hidden_state, which additionally passes through post_layernorm. This is the convention the distillation loss was defined against, and recent Transformers versions report the post-norm tensor in hidden_states[-1] on the official DINOv3 teacher, so the two APIs differ at that one index.

Use hidden_states[i] for anything layer-wise — ALOEv2 supervises blocks n/3, 2n/3 and n, where its cosine similarity to the DINOv3 teacher's corresponding pre-norm features is 0.98–0.99. post_layernorm was not part of the loss, so last_hidden_state is noticeably less aligned (0.78–0.86).

Usage — ImageNet-1k classification and explanations (-in1k-lp only)

from PIL import Image
from transformers import AutoImageProcessor, AutoModelForImageClassification

repo_id = "rmaser/aloe-v2-dinov3-base-in1k-lp"
processor = AutoImageProcessor.from_pretrained(repo_id, trust_remote_code=True)
model = AutoModelForImageClassification.from_pretrained(repo_id, trust_remote_code=True)
model.eval()

image = Image.open("image.jpg").convert("RGB")
pixel_values = processor(images=image, return_tensors="pt").pixel_values

result = model.explain(pixel_values, idx=None)
class_idx = int(result["explained_class_idx"][0])
print(f"Predicted ImageNet-1k class index: {class_idx}")

rgba = (result["explanation"][0] * 255).astype("uint8")
Image.fromarray(rgba).save("explanation.png")

idx=None explains the predicted class. Pass an ImageNet-1k class index to idx to explain a specific class instead. Do not wrap model.explain(...) in torch.inference_mode(): generating the attribution requires input gradients.

Notes

ALOE models use custom B-cos-aware Transformers code, so loading requires trust_remote_code=True. Runtime code: rmaser/aloe-arch.

Links

• Code: github.com/rmaser/ALOE
• Original ALOE models: ALOE on Hugging Face
• Shared runtime code: rmaser/aloe-arch
• Paper: Align Once to Explain (CVPR 2026)

Citation

@inproceedings{maser2026align,
  title = {Align Once to Explain: Feature Alignment for Scalable B-cosification of Foundational Vision Transformers},
  author = {Maser, Raphael and Gairola, Siddhartha and Rao, Sukrut and Schiele, Bernt},
  booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
  year = {2026},
  note = {Poster}
}

License

This model is released under the Creative Commons Attribution-NonCommercial ShareAlike 4.0 International License. The methods described in this work are patent pending.