RationalRewards-Ed

About

What RationalRewards-Ed is

TLDR: this is a reasoning reward model that supports image editing, from the following paper. RationalRewards: Reasoning Rewards Scale Visual Generation Both Training and Test Time RationalRewards is a reasoning-based reward model and toolkit for visual generation. Instead of reducing preference into one opaque scalar, it generates explicit multi-dimensional critiques before scoring, turning reward models from passive evaluators into active optimization interfaces. About the name: "Rational" means being reasonable, sensible, in Chinese, 理性的 RationalRewards supports optimization in complementary spaces: - train-time optimization through RL with structured, interpretable reward signals, and - test-time optimization through a Generate-Critique-Refine loop without parameter updates. Key Results Instantiated via PARROT on a Qwen3-VL-Instruct-8B backbone, RationalRewards achieves state-of-the-art preference prediction among open-source reward models and remains competitive with Gemini-2.5-Pro. As an RL reward, it consistently improves generators beyond scalar baselines across both text-to-image and image-editing tasks. Most interestingly, RationalRewards' test-time prompt tuning, requiring no parameter updates, matches or exceeds RL-based fine-tuning on several benchmarks. Train-time RL and test-time prompt tuning with RationalRewards across visual generation benchmarks. Why Reasoning Rewards? Most reward models collapse instruction following, visual quality, composition, and plausibility into one scalar. This removes the structure of human judgment and often leads to brittle optimization. RationalRewards keeps those dimensions explicit so generators receive semantically grounded feedback about what to fix and why. Why do reasoning rewards resist reward hacking? Scalar rewards are vulnerable to reward hacking because they collapse rich judgment into one number that can rise even when outputs do not truly improve. RationalRewards introduces an implicit regularization: before giving scores, it must produce coherent, multi-dimensional critiques tied to concrete evaluation axes. This constrains optimization to evidence-backed reasoning and improves the monotonic relations

Summarised from the published model card. Read the full card on the HuggingFace links below.

Specifications

What it is

How it works

How language models work

Variants

Sizes & precisions

Open weights ship in several sizes and precisions. One page, all the variants — pick the one that fits your GPU. VRAM figures are estimates from model size.

Using it via the API

Call it like any OpenAI endpoint

Once AxForge deploys rationalrewards-ed for you, it answers on the OpenAI-compatible API — the same base URL and keys as every other model. (rationalrewards-ed below is illustrative; you get the exact model name on deployment.)

$ curl -sS https://api.axforge.ai/v1/chat/completions \
  -H "Authorization: Bearer $AXFORGE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"rationalrewards-ed","messages":[{"role":"user","content":"Hello"}]}'

Details

Languages, data & research

Papers

Licence

Open weights

Sources

Weights & code

Explore

More language models