InfiGUIAgent: A Multimodal Generalist GUI Agent with Native Reasoning and Reflection

Yuhang Liu, Pengxiang Li, Zishu Wei, Congkai Xie, Xueyu Hu, Xinchen Xu, Shengyu Zhang, Xiaotian Han, Hongxia Yang, Fei Wu · 2025

Read InfiGUIAgent: A Multimodal Generalist GUI Agent with Native Reasoning and Reflection by Yuhang Liu, Pengxiang Li, Zishu Wei, Congkai Xie, Xueyu Hu, Xinchen Xu, Shengyu Zhang, Xiaotian Han, Hongxia Yang, Fei Wu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2501-04575.txt.

Description

Yuhang Liu and co-authors present InfiGUIAgent, a multimodal agent that works on graphical interfaces, in a 2025 paper. Such agents, powered by large multimodal models, promise task automation on computers and phones. Agents built so far struggle with multi step reasoning and lean on text annotations, which limits how well they work on live screens.

Training runs in two stages of supervised fine tuning. The first stage builds core skills in interface understanding and grounding. The second adds hierarchical reasoning and an expectation reflection skill from synthesized data, so the reasoning is native to the agent rather than bolted on. Results on several interface benchmarks are competitive, the authors report.

Catalog record

CREATORYuhang Liu, Pengxiang Li, Zishu Wei, Congkai Xie, Xueyu Hu, Xinchen Xu, Shengyu Zhang, Xiaotian Han, Hongxia Yang, Fei Wu
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE26 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2501-04575

Browse the full library · About the Tower of Babel