Towards GUI Agents: Vision-Language Diffusion Models for GUI Grounding

Shrinidhi Kumbhar, Haofu Liao, Srikar Appalaraju, Kunwar Yashraj Singh · 2026

Read Towards GUI Agents: Vision-Language Diffusion Models for GUI Grounding by Shrinidhi Kumbhar, Haofu Liao, Srikar Appalaraju, Kunwar Yashraj Singh (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-26211.txt.

Description

Kumbhar and co-authors test whether discrete diffusion vision-language models can locate elements on a screen. They adapt the LLaDA-V model to predict actions and bounding boxes for web, desktop, and mobile screens. A hybrid masking schedule that mixes linear and fixed masking lifts step success by up to 6.1 points. More training data from varied GUI domains cuts delay by about 1.3 seconds. Accuracy also rises by an average of 20 points across tests.

Catalog record

CREATORShrinidhi Kumbhar, Haofu Liao, Srikar Appalaraju, Kunwar Yashraj Singh
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE45985
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.26211

Browse the full library · About the Tower of Babel