reka-edge-2603

Key features

• Faster and more token-efficient than similarly sized VLMs
• Strong benchmark performance across VQA-v2, RefCOCO, MLVU, MMVU and Mobile Actions (see below)
• Support for vLLM (see plugin)
• Open weights license: the model can be used commercially if you make less than $1 million USD of revenue a year

Benchmarks and metrics

*Gemini 3 Pro measured via API call; other models measured with local inference.

Quick Start

llama.cpp

To get started:

1. Use the weights from repo
2. Build the necessary artifacts from llama.cpp repo

cmake -B build
cmake --build build --target llama-server -j
cmake --build build --target llama-quantize -j

3. Run the GGUF conversion script (convert_reka_vlm_to_gguf.py) from the llama.cpp repo root

python3 convert_reka_vlm_to_gguf.py /path/to/reka/weights \
  --outfile /path/to/reka-text-f16.gguf \
  --outtype f16

# Export the vision encoder
python3 convert_reka_vlm_to_gguf.py /path/to/reka/weights \
  --mmproj \
  --outfile /path/to/reka-mmproj-f16.gguf \
  --outtype f16

4. (optional) Use the quantization scripts (quantize_reka_...) for simple quantizations of the model

# Example usage for text decoder quantization
bash inference/hf_release/quantize_reka_q4_last8_q8.sh /path/to/reka-text-f16.gguf /path/to/reka-text-q4_last8_q8.gguf

5. Run llama-server

./build/bin/llama-server -m /path/to/reka-text-f16.gguf \
  --mmproj /path/to/reka-mmproj-f16.gguf \
  -t 8 -c 2048 --host 0.0.0.0 --port 8080 --reasoning off \

One note: the model does not currently support reasoning, so we run llama-server with --reasoning off.

🤗 Transformers (macOS)

The easiest way to run the model is with the included example.py script. It uses PEP 723 inline metadata so uv resolves dependencies automatically — no manual install step:

uv run example.py --image media/hamburger.jpg --prompt "What is in this image?"

Requirements

Edge Deployment Devices

• Mac devices with Apple Silicon
  • OS: macOS 13+
  • Minimum: 24 GB memory
  • Recommended: 32 GB+ memory
• Linux and Windows Subsystem for Linux (WSL) PCs
  • Minimum: 24 GB GPU and 24 GB+ system memory
  • Recommended: 32 GB+ GPU and 32 GB+ system memory
• Nvidia Robotics & Edge AI systems
  • Jetson Thor
  • Jetson AGX Orin (both 32 GB and 64 GB variants)

Custom Deployment Options

With quantization, Reka Edge can also be run on:

• Jetson Orin Nano
• Samsung S25
• Qualcomm Snapdragon XR2 Gen 3 devices
• Apple iPhone, iPad, and Vision Pro

Reach out for support deploying Reka Edge to a custom edge compute platform.

Software Requirements

• Python: 3.12+
• uv (recommended) — handles dependencies automatically

Inline snippet

If you prefer not to use the script, install dependencies manually and paste the code below:

uv pip install "transformers==4.57.3" torch torchvision pillow tiktoken imageio einops av

import torch
from PIL import Image
from transformers import AutoModelForImageTextToText, AutoProcessor

model_id = "RekaAI/reka-edge-2603"

# Load processor and model
processor = AutoProcessor.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForImageTextToText.from_pretrained(
    model_id,
    trust_remote_code=True,
    torch_dtype=torch.float16,
).eval()

# Move to MPS (Apple Silicon GPU)
device = torch.device("mps")
model = model.to(device)

# Prepare an image + text query
image_path = "media/hamburger.jpg"  # included in the model repo
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "image": image_path},
            {"type": "text", "text": "What is in this image?"},
        ],
    }
]

# Tokenize using the chat template
inputs = processor.apply_chat_template(
    messages,
    tokenize=True,
    add_generation_prompt=True,
    return_tensors="pt",
    return_dict=True,
)

# Move tensors to device
for key, val in inputs.items():
    if isinstance(val, torch.Tensor):
        if val.is_floating_point():
            inputs[key] = val.to(device=device, dtype=torch.float16)
        else:
            inputs[key] = val.to(device=device)

# Generate
with torch.inference_mode():
    # Stop on  token (end-of-turn) in addition to default EOS
    sep_token_id = processor.tokenizer.convert_tokens_to_ids("")
    output_ids = model.generate(
        **inputs,
        max_new_tokens=256,