WorldGUI: An Interactive Benchmark for Desktop GUI Automation from Any Starting Point
Henry Hengyuan Zhao, Kaiming Yang, Wendi Yu, Difei Gao, Mike Zheng Shou · 2025
Read WorldGUI: An Interactive Benchmark for Desktop GUI Automation from Any Starting Point by Henry Hengyuan Zhao, Kaiming Yang, Wendi Yu, Difei Gao, Mike Zheng Shou (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-08047.txt.
Description
Zhao and co-authors introduce WorldGUI in 2025, a benchmark that tests computer-using agents from messy starting points. Most tests start apps in a clean default state. Real users ask for help mid-task, with settings changed and steps half done.
The benchmark covers ten common desktop and web applications. Each task comes in systematically varied initial states that mimic real work in progress. Even top agents lost much of their performance under these non-default starts, showing fragile planning. The authors also give a simple agent framework that critiques its own plan at three stages, which improved reliability. They release the code and data for other teams.
Catalog record
| CREATOR | Henry Hengyuan Zhao, Kaiming Yang, Wendi Yu, Difei Gao, Mike Zheng Shou |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 62 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2502-08047