Model reference · open weights
bloomz-petals is an open-weight language model from bigscience. AxForge deploys and operates it for you on dedicated EU-owned hardware — with the licence handled where one is required.
Available as managed deployment — configured and operated for you on dedicated EU hardware, quoted per deployment.
What it is
| Maker | bigscience |
|---|---|
| Type | Language models |
| Task | Text gen |
| Runs with | transformers |
| Released | 2023-01-16 |
| Popularity | 37 downloads / month |
| Licence | Unknown |
About
This model is a version of bigscience/bloomz post-processed to be run at home using the Petals swarm.
Please check out:
We provide minimal code examples below.
from petals import DistributedBloomForCausalLM
model = DistributedBloomForCausalLM.from_pretrained("bigscience/bloomz-petals")
# Embeddings & prompts are on your device, BLOOM blocks are distributed across the Internet
inputs = tokenizer("A cat sat", return_tensors="pt")["input_ids"]
outputs = model.generate(inputs, max_new_tokens=5)
print(tokenizer.decode(outputs[0])) # A cat sat on a mat...
python -m petals.cli.run_server bigscience/bloomz-petals
From the published model card. Full card on the HuggingFace links in the sidebar.
Using it via the API
Once AxForge deploys bloomz-petals for you, it answers on the OpenAI-compatible API — the same base URL and keys as every other model. (bloomz-petals below is illustrative; you get the exact model name on deployment.)
$ curl -sS https://api.axforge.ai/v1/chat/completions \
-H "Authorization: Bearer $AXFORGE_API_KEY" \
-H "Content-Type: application/json" \
-d '{"model":"bloomz-petals","messages":[{"role":"user","content":"Hello"}]}'
Create an account — your API key is available in the console. 5M tokens/month currently included with every new account at launch.