Wan2.2-Fun-Reward-LoRAs

About

What Wan2.2-Fun-Reward-LoRAs is

Wan2.2-Fun-Reward-LoRAs Introduction We explore the Reward Backpropagation technique <sup1 2</sup to optimized the generated videos by Wan2.2-Fun for better alignment with human preferences. We provide the following pre-trained models (i.e. LoRAs) along with the training script. You can use these LoRAs to enhance the corresponding base model as a plug-in or train your own reward LoRA. For more details, please refer to our GitHub repo. [!NOTE] We found that, MPS reward LoRA for the low-noise model converges significantly more slowly than on the other models, and may not deliver satisfactory results. Therefore, for the low-noise model, we recommend using HPSv2.1 reward LoRA. Demo Wan2.2-Fun-A14B-InP A panda eats bamboo while a monkey swings from branch to branch A dog runs through a field while a cat climbs a tree A penguin waddles on the ice, a camel treks by Pig with wings flying above a diamond mountain [!NOTE] The above test prompts are from <a href="https://github.com/KaiyueSun98/T2V-CompBench"T2V-CompBench</a and expanded into detailed prompts by Llama-3.3. Videos are generated with HPSv2.1 Reward LoRA weight 0.5 and MPS Reward LoRA weight 0.5. Quick Start Set lorapath along with loraweight for the low noise reward LoRA, while specifying lorahighpath and lorahighweight for high noise reward LoRA in examples/wan2.2fun/predictt2v.py. Training Please refer to READMETRAINREWARD.md Limitations 1. We observe after training to a certain extent, the reward continues to increase, but the quality of the generated videos does not further improve. The model trickly learns some shortcuts (by adding artifacts in the background, i.e., adversarial patches) to increase the reward. 2. Currently, there is still a lack of suitable preference models for video generation. Directly using image preference models cannot evaluate preferences along the temporal dimension (such as dynamism and consistency). Further more, We find using image preference models leads to a decrease in the dynamism of generated videos. Although this can be mitigated by computing the reward using only the first frame of the decoded video, the impact still persists. Reference

Summarised from the published model card. Read the full card on the HuggingFace links below.

Specifications

What it is

How it works

How video models work

Variants

Sizes & precisions

Open weights ship in several sizes and precisions. One page, all the variants — pick the one that fits your GPU. VRAM figures are estimates from model size.

Using it via the API

Call it like any OpenAI endpoint

Once AxForge deploys wan2-2-fun-reward-loras for you, it answers on the OpenAI-compatible API — the same base URL and keys as every other model. (wan2-2-fun-reward-loras below is illustrative; you get the exact model name on deployment.)

$ curl -sS https://api.axforge.ai/v1/videos/generations \
  -H "Authorization: Bearer $AXFORGE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"wan2-2-fun-reward-loras","prompt":"a drone shot over a forest"}'

Details

Languages, data & research

Papers

Licence

Open weights

Sources

Weights & code

Explore

More video models