GUI-R1: A Generalist R1-Style Vision-Language Action Model For GUI Agents

Run Luo Lu Wang Wanwei He Xiaobo Xia · 2025

Read GUI-R1: A Generalist R1-Style Vision-Language Action Model For GUI Agents by Run Luo Lu Wang Wanwei He Xiaobo Xia (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-10458.txt.

Description

The authors present GUI-R1, a vision-language model trained to operate software interfaces as an agent. Most interface agents learn by supervised fine-tuning, which demands huge training sets and still generalizes poorly to unseen screens. Inspired by reinforcement fine-tuning in reasoning models, the team trains with rewards instead. Their framework rests on a unified action space that describes interface actions by shared rules across platforms.

Training uses a small, carefully curated set of about three thousand examples spanning Windows, Linux, MacOS, Android, and the web. A policy optimization method updates the model from its own trial results. Across eight benchmarks on mobile, desktop, and web, the model beats prior leading systems while using only a tiny fraction of their data. The authors argue reinforcement learning with a unified action model holds large promise for real interface tasks.

Catalog record

CREATORRun Luo Lu Wang Wanwei He Xiaobo Xia
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE31 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2504-10458

Browse the full library · About the Tower of Babel