specter2

About

What specter2 is

SPECTER2 SPECTER2 is the successor to SPECTER and is capable of generating task specific embeddings for scientific tasks when paired with adapters. This is the base model to be used along with the adapters. Given the combination of title and abstract of a scientific paper or a short texual query, the model can be used to generate effective embeddings to be used in downstream applications. Note:For general embedding purposes, please use allenai/specter2. To get the best performance on a downstream task type please load the associated adapter with the base model as in the example below. Dec 2023 Update: Model usage updated to be compatible with latest versions of transformers and adapters (newly released update to adapter-transformers) libraries. Aug 2023 Update: 1. The SPECTER2 Base and proximity adapter models have been renamed in Hugging Face based upon usage patterns as follows: 2. We have a parallel version (termed aug2023refresh) where the base transformer encoder version is pre-trained on a collection of newer papers (published after 2018). However, for benchmarking purposes, please continue using the current version. An adapter for the allenai/specter2base model that was trained on the allenai/scirepeval dataset. This adapter was created for usage with the adapters library. Model Details Model Description SPECTER2 has been trained on over 6M triplets of scientific paper citations, which are available here. Post that it is trained with additionally attached task format specific adapter modules on all the SciRepEval training tasks. Task Formats trained on: - Classification - Regression - Proximity (Retrieval) - Adhoc Search It builds on the work done in SciRepEval: A Multi-Format Benchmark for Scientific Document Representations and we evaluate the trained model on this benchmark as well. - Developed by: Amanpreet Singh, Mike D'Arcy, Arman Cohan, Doug Downey, Sergey Feldman - Shared by : Allen AI - Model type: bert-base-uncased + adapters - License: Apache 2.0 - Finetuned from model: allenai/scibert. Model Sources - Repository: https://github.com/allenai/SPECTER2 - Paper: https://api.semanticscholar.org/CorpusID:254018137 - Demo: Usage Uses Direct Use Proxim

Summarised from the published model card. Read the full card on the HuggingFace links below.

Specifications

What it is

How it works

How embedding models work

Variants

Sizes & precisions

Open weights ship in several sizes and precisions. One page, all the variants — pick the one that fits your GPU. VRAM figures are estimates from model size.

Using it via the API

Call it like any OpenAI endpoint

Once AxForge deploys specter2 for you, it answers on the OpenAI-compatible API — the same base URL and keys as every other model. (specter2 below is illustrative; you get the exact model name on deployment.)

$ curl -sS https://api.axforge.ai/v1/embeddings \
  -H "Authorization: Bearer $AXFORGE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"specter2","input":"text to embed"}'

Details

Languages, data & research

Licence

Open weights

Sources

Weights & code

Explore

More embedding models