Model reference · open weights
RITA_s is an open-weight language model from lightonai. AxForge deploys and operates it for you on dedicated EU-owned hardware — with the licence handled where one is required.
Available as managed deployment — configured and operated for you on dedicated EU hardware, quoted per deployment.
What it is
| Released by | lightonai |
|---|---|
| Type | Language models |
| Task | Text gen |
| Parameters (lead) | 85M |
| Context | 1k tokens |
| Runs with | transformers |
| Released | 2022-04-25 |
| Popularity | 627 downloads / month |
| Licence | Unknown |
About
RITA is a family of autoregressive protein models, developed by a collaboration of Lighton, the OATML group at Oxford, and the Debbie Marks Lab at Harvard.
| Model | #Params | d_model | layers | lm loss uniref-100 |
|---|---|---|---|---|
| Small | 85M | 768 | 12 | 2.31 |
| Medium | 300M | 1024 | 24 | 2.01 |
| Large | 680M | 1536 | 24 | 1.82 |
| XLarge | 1.2B | 2048 | 24 | 1.70 |
For full results see our preprint: https://arxiv.org/abs/2205.05789
Instantiate a model like so:
from transformers import AutoModel, AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("lightonai/RITA_s, trust_remote_code=True")
tokenizer = AutoTokenizer.from_pretrained("lightonai/RITA_s")
for generation we support pipelines:
from transformers import pipeline
rita_gen = pipeline('text-generation', model=model, tokenizer=tokenizer)
sequences = rita_gen("MAB", max_length=20, do_sample=True, top_k=950, repetition_penalty=1.2,
num_return_sequences=2, eos_token_id=2)
for seq in sequences:
print(f"seq: {seq['generated_text'].replace(' ', '')}")
@article{hesslow2022rita,
title={RITA: a Study on Scaling Up Generative Protein Sequence Models},
author={Hesslow, Daniel and Zanichelli, Niccol{\'o} and Notin, Pascal and Poli, Iacopo and Marks, Debora},
journal={arXiv preprint arXiv:2205.05789},
year={2022}
}From the published model card. Full card on the HuggingFace links in the sidebar.
Using it via the API
Once AxForge deploys rita-s for you, it answers on the OpenAI-compatible API — the same base URL and keys as every other model. (rita-s below is illustrative; you get the exact model name on deployment.)
$ curl -sS https://api.axforge.ai/v1/chat/completions \
-H "Authorization: Bearer $AXFORGE_API_KEY" \
-H "Content-Type: application/json" \
-d '{"model":"rita-s","messages":[{"role":"user","content":"Hello"}]}'
Create an account — your API key is available in the console. 3M free tokens every 30 days with every new account.