WorldGUI: An Interactive Benchmark for Desktop GUI Automation from Any Starting Point

Henry Hengyuan Zhao, Kaiming Yang, Wendi Yu, Difei Gao, Mike Zheng Shou · 2025

Read WorldGUI: An Interactive Benchmark for Desktop GUI Automation from Any Starting Point by Henry Hengyuan Zhao, Kaiming Yang, Wendi Yu, Difei Gao, Mike Zheng Shou (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-08047.txt.

Description

Zhao and co-authors introduce WorldGUI in 2025, a benchmark that tests computer-using agents from messy starting points. Most tests start apps in a clean default state. Real users ask for help mid-task, with settings changed and steps half done.

The benchmark covers ten common desktop and web applications. Each task comes in systematically varied initial states that mimic real work in progress. Even top agents lost much of their performance under these non-default starts, showing fragile planning. The authors also give a simple agent framework that critiques its own plan at three stages, which improved reliability. They release the code and data for other teams.

Catalog record

CREATORHenry Hengyuan Zhao, Kaiming Yang, Wendi Yu, Difei Gao, Mike Zheng Shou
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE62 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2502-08047

Browse the full library · About the Tower of Babel