Model reference · open weights
gemma-2 is an open-weight language model from Efficient-Large-Model, listed in the AxForge catalogue. AxForge can bring it up on EU-owned hardware for you on request — with the licence handled where one is required.
About
Gemma 2 model card Model Page: Gemma Resources and Technical Documentation: [Responsible Generative AI Toolkit][rai-toolkit] [Gemma on Kaggle][kaggle-gemma] [Gemma on Vertex Model Garden][vertex-mg-gemma2] Terms of Use: [Terms][terms] Authors: Google Model Information Summary description and brief definition of inputs and outputs. Description Gemma is a family of lightweight, state-of-the-art open models from Google, built from the same research and technology used to create the Gemini models. They are text-to-text, decoder-only large language models, available in English, with open weights for both pre-trained variants and instruction-tuned variants. Gemma models are well-suited for a variety of text generation tasks, including question answering, summarization, and reasoning. Their relatively small size makes it possible to deploy them in environments with limited resources such as a laptop, desktop or your own cloud infrastructure, democratizing access to state of the art AI models and helping foster innovation for everyone. Usage Below we share some code snippets on how to get quickly started with running the model. First, install the Transformers library with: Then, copy the snippet from the section that is relevant for your usecase. Running with the pipeline API Running the model on a single / multi GPU You can ensure the correct chat template is applied by using tokenizer.applychattemplate as follows: Running the model on a GPU using different precisions The native weights of this model were exported in bfloat16 precision. You can also use float32 if you skip the dtype, but no precision increase will occur (model weights will just be upcasted to float32). See examples below. Upcasting to torch.float32 Running the model through a CLI The local-gemma repository contains a lightweight wrapper around Transformers for running Gemma 2 through a command line interface, or CLI. Follow the installation instructions for getting started, then launch the CLI through the following command: Quantized Versions through bitsandbytes Using 8-bit precision (int8) Using 4-bit precision Advanced Usage Torch compile Torch compile is a method for speeding-up the inference of Py
Summarised from the published model card. Read the full card on the HuggingFace links below.
Specifications
| Maker | Efficient-Large-Model |
|---|---|
| Type | Language models |
| Parameters (lead) | 2.6B |
| Variants | 1 |
| Runs with | transformers |
| Based on | google/gemma-2-2b |
| Released | 2024-12-12 |
| Popularity | 149k downloads / month |
| Likes | 3 |
| Licence | Open, with conditions |
How it works
Variants
Open weights ship in several sizes and precisions. One page, all the variants — pick the one that fits your GPU. VRAM figures are estimates from model size.
| Variant | Params | Precision | VRAM | Fits 16 GB | Weights |
|---|---|---|---|---|---|
| gemma-2-2b-it | 2.6B | BF16 | ~6 GB | ✓ | Weights ↗ |
Using it via the API
Once AxForge deploys efficient-large-model-gemma-2 for you, it answers on the OpenAI-compatible API — the same base URL and keys as every other model. (efficient-large-model-gemma-2 below is illustrative; you get the exact model name on deployment.)
$ curl -sS https://api.axforge.ai/v1/chat/completions \
-H "Authorization: Bearer $AXFORGE_API_KEY" \
-H "Content-Type: application/json" \
-d '{"model":"efficient-large-model-gemma-2","messages":[{"role":"user","content":"Hello"}]}'
Details
Tags
Papers
Licence
Open weights under gemma, which carries conditions (e.g. attribution or an acceptable-use clause). Worth a read before production use — we can walk you through it. Read the licence ↗