PE-Core-L14-336

About

What PE-Core-L14-336 is

Model Details [\[📃 Tech Report\]](https://arxiv.org/abs/2504.13181) [\[📂 Github\]](https://github.com/facebookresearch/perceptionmodels/) Perception Encoder (PE) is a state-of-the-art encoder for image and video understanding trained via simple vision-language learning. It was introduced in "[Perception Encoder: The best visual embeddings are not at the output of the network](https://ai.meta.com/research/publications/perception-encoder-the-best-visual-embeddings-are-not-at-the-output-of-the-network/)". Model Developer: Meta Model Overview: Perception Encoder (PE) is a family of large-scale vision encoder models with state-of-the-art performance on a large variety of vision tasks. By using a robust contrastive pretraining recipe and finetuning on synthetically aligned videos, PE not only outperforms all existing models on classification and retrieval, but it also internally produces strong, general features that scale for downstream tasks. PE unlocks the ability for large-scale contrastive pretraining to transfer to downstream tasks with alignment tuning to capitalize on those general features. Perception Encoder: Core PE core is our base model trained with our robust image pretraining schedule and finetuned on the data generated by our synthetic video data engine. Model Configurations PE core curently comes in 3 sizes. PE core G is the main checkpoint, with L and B models distilled from it. All PE core models use an attention pooling block with 8 heads on top of the vision tower. The L and B models additionally have a class token for global aggregation. See the paper for more details. Model Performance PE core obtains extremely strong results across the board on zero-shot image classification and retrieval as well as zero-shot video classification and retrieval. We present a sample of its performance across those domains below. PE core performs particularly well on the hard benchmarks such as ObjectNet and ImageNet-A. How to use Model loading code We provide the model loading code in https://github.com/facebookresearch/perceptionmodels This will install an editable version of repo, allowing you to make changes to the code without needing to reinstall the package

Summarised from the published model card. Read the full card on the HuggingFace links below.

Specifications

What it is

How it works

How embedding models work

Variants

Sizes & precisions

Open weights ship in several sizes and precisions. One page, all the variants — pick the one that fits your GPU. VRAM figures are estimates from model size.

Using it via the API

Call it like any OpenAI endpoint

Once AxForge deploys pe-core-l14-336 for you, it answers on the OpenAI-compatible API — the same base URL and keys as every other model. (pe-core-l14-336 below is illustrative; you get the exact model name on deployment.)

$ curl -sS https://api.axforge.ai/v1/embeddings \
  -H "Authorization: Bearer $AXFORGE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"pe-core-l14-336","input":"text to embed"}'

Details

Languages, data & research

Papers

Licence

Open weights

Sources

Weights & code

Explore

More embedding models