ScreenExplorer: Training a Vision–Language Model for Diverse Exploration in Open GUI World
Runliang Niu, Jinglong Ji, Yi Chang, Qi Wang · 2025
Read ScreenExplorer: Training a Vision–Language Model for Diverse Exploration in Open GUI World by Runliang Niu, Jinglong Ji, Yi Chang, Qi Wang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-19095.txt.
Description
Runliang Niu, Jinglong Ji, Yi Chang, and Qi Wang train a vision-language model to explore computer interfaces on its own, in a paper from 2025. Screen agents usually memorize fixed task data and fail in new apps. Learning by exploring live environments could break that dependence on hand-made datasets.
Training ran inside real, changing graphical interfaces with a group-based reinforcement method. A world-model curiosity reward pushed the agent through the awkward cold-start phase when nothing works yet. Distilling its own experience streams further sharpened exploration. The trained model adapted to new environments better and kept exploring longer than static counterparts.
Catalog record
| CREATOR | Runliang Niu, Jinglong Ji, Yi Chang, Qi Wang |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 59 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-19095