Building Autonomous GUI Navigation via Agentic-Q Estimation and Step-Wise Policy Optimization
Yibo Wang, Guangda Huzhang, Yuwei Hu, Yu Xia, Shiyin Lu, Qing-Guo Chen, Zhao Xu, Weihua Luo, Kaifu Zhang, Lijun Zhang, National Key Laboratory for Novel Software Technology, Nanjing University, Ovis Team, Alibaba Group · 2026
Read Building Autonomous GUI Navigation via Agentic-Q Estimation and Step-Wise Policy Optimization by Yibo Wang, Guangda Huzhang, Yuwei Hu, Yu Xia, Shiyin Lu, Qing-Guo Chen, Zhao Xu, Weihua Luo, Kaifu Zhang, Lijun Zhang, National Key Laboratory for Novel Software Technology, Nanjing University, Ovis Team, Alibaba Group (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-13653.txt.
Description
Graphical interface agents must cope with screens and apps that keep changing, which makes data collection and policy training costly in real deployments. This framework centers on multimodal language models and pairs two components. Agentic-Q estimation learns a Q-model scoring each action's contribution to task completion step by step, and step-wise policy optimization trains the policy by reinforcement learning on those scored steps.
All state-action trajectories come from the policy itself, keeping data costs manageable, and policy updates are decoupled from the environment for stable efficient optimization. The framework gave Ovis2.5-9B strong interface interaction skills, with remarkable results on navigation and grounding benchmarks, even beating larger contenders. Yibo Wang leads the work with nine co-authors.
Catalog record
| CREATOR | Yibo Wang, Guangda Huzhang, Yuwei Hu, Yu Xia, Shiyin Lu, Qing-Guo Chen, Zhao Xu, Weihua Luo, Kaifu Zhang, Lijun Zhang, National Key Laboratory for Novel Software Technology, Nanjing University, Ovis Team, Alibaba Group |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 43 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2602-13653