Model reference · open weights
bloom-7b1-petals is an open-weight language model from bigscience. AxForge deploys and operates it for you on dedicated EU-owned hardware — with the licence handled where one is required.
Available as managed deployment — configured and operated for you on dedicated EU hardware, quoted per deployment.
What it is
| Maker | bigscience |
|---|---|
| Type | Language models |
| Task | Text gen |
| Parameters (lead) | 1.0B |
| Runs with | transformers |
| Released | 2023-01-22 |
| Popularity | 204 downloads / month |
| Licence | Unknown |
About
This model is a version of bigscience/bloom-7b1 post-processed to be run at home using the Petals swarm.
Note: Petals is developed to run 100B+ models like the full-scale BLOOM or BLOOMZ. This model is provided for testing purposes only. It may be more efficient to run the original version of it locally.
Please check out:
We provide minimal code examples below.
from petals import DistributedBloomForCausalLM
model = DistributedBloomForCausalLM.from_pretrained("bigscience/bloom-7b1-petals")
# Embeddings & prompts are on your device, BLOOM blocks are distributed across the Internet
inputs = tokenizer("A cat sat", return_tensors="pt")["input_ids"]
outputs = model.generate(inputs, max_new_tokens=5)
print(tokenizer.decode(outputs[0])) # A cat sat on a mat...
python -m petals.cli.run_server bigscience/bloom-7b1-petals
From the published model card. Full card on the HuggingFace links in the sidebar.
Using it via the API
Once AxForge deploys bloom-7b1-petals for you, it answers on the OpenAI-compatible API — the same base URL and keys as every other model. (bloom-7b1-petals below is illustrative; you get the exact model name on deployment.)
$ curl -sS https://api.axforge.ai/v1/chat/completions \
-H "Authorization: Bearer $AXFORGE_API_KEY" \
-H "Content-Type: application/json" \
-d '{"model":"bloom-7b1-petals","messages":[{"role":"user","content":"Hello"}]}'
Create an account — your API key is available in the console. 5M tokens/month currently included with every new account at launch.