GTA1

Reinforcement learning (RL) (e.g., GRPO) helps with grounding because of its inherent objective alignment—rewarding successful clicks—rather than encouraging long textual Chain-of-Thought (CoT) reasoning. Unlike approaches that rely heavily on verbose CoT reasoning, GRPO directly incentivizes actionable and grounded responses. Based on findings from our blog, we share state-of-the-art GUI grounding models trained using GRPO.

Grounding Performance

We follow the standard evaluation protocol and benchmark our model on three challenging datasets. Our method consistently achieves the best results among all open-source model families. Below are the comparative results:

Note:

• Model size is indicated in billions (B) of parameters.
• A dash (—) denotes results that are currently unavailable.
• A superscript asterisk (﹡) denotes our evaluated result.
• UI-TARS-1.5 7B, OpenCUA-7B, and OpenCUA-32B are applied as our baseline models.
• ∆ indicates the performance improvement (∆) of our model compared to its baseline.

Agent Performance

OSWorld and OSWorld-Verified Benchmarks

We evaluate our models on the OSWorld and OSWorld-Verified benchmarks following the standard evaluation protocol. The results demonstrate strong performance across both datasets.

Note: A dash (—) indicates unavailable results.

WindowsAgentArena Benchmark

We also evaluate our models on the WindowsAgentArena benchmark, demonstrating strong performance in Windows-specific GUI automation tasks.

Note: A dash (—) indicates unavailable results.

Inference

Below is a code snippet demonstrating how to run inference using a trained model.

from transformers import AutoTokenizer, AutoImageProcessor
from transformers.models.qwen2_vl.image_processing_qwen2_vl_fast import smart_resize
from PIL import Image
from io import BytesIO
import base64
import re
from vllm import LLM, SamplingParams

instruction="click start"
image_path="example.png"

CLICK_REGEXES = [