Building Autonomous GUI Navigation via Agentic-Q Estimation and Step-Wise Policy Optimization

Yibo Wang, Guangda Huzhang, Yuwei Hu, Yu Xia, Shiyin Lu, Qing-Guo Chen, Zhao Xu, Weihua Luo, Kaifu Zhang, Lijun Zhang, National Key Laboratory for Novel Software Technology, Nanjing University, Ovis Team, Alibaba Group · 2026

Read Building Autonomous GUI Navigation via Agentic-Q Estimation and Step-Wise Policy Optimization by Yibo Wang, Guangda Huzhang, Yuwei Hu, Yu Xia, Shiyin Lu, Qing-Guo Chen, Zhao Xu, Weihua Luo, Kaifu Zhang, Lijun Zhang, National Key Laboratory for Novel Software Technology, Nanjing University, Ovis Team, Alibaba Group (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-13653.txt.

Description

Graphical interface agents must cope with screens and apps that keep changing, which makes data collection and policy training costly in real deployments. This framework centers on multimodal language models and pairs two components. Agentic-Q estimation learns a Q-model scoring each action's contribution to task completion step by step, and step-wise policy optimization trains the policy by reinforcement learning on those scored steps.

All state-action trajectories come from the policy itself, keeping data costs manageable, and policy updates are decoupled from the environment for stable efficient optimization. The framework gave Ovis2.5-9B strong interface interaction skills, with remarkable results on navigation and grounding benchmarks, even beating larger contenders. Yibo Wang leads the work with nine co-authors.

Catalog record

CREATORYibo Wang, Guangda Huzhang, Yuwei Hu, Yu Xia, Shiyin Lu, Qing-Guo Chen, Zhao Xu, Weihua Luo, Kaifu Zhang, Lijun Zhang, National Key Laboratory for Novel Software Technology, Nanjing University, Ovis Team, Alibaba Group
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE43 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602-13653

Browse the full library · About the Tower of Babel