UI-Copilot: Advancing Long-Horizon GUI Automation via Tool-Integrated Policy Optimization

Zhengxi Lu 1 , Fei Tang 1 , Guangyi Liu 1 , Kaitao Song 2 , Xu Tan 3 , Jin Ma 3 Wenqi Zhang 1 , Weiming Lu 1 , Jun Xiao 1 , Yueting Zhuang 1 , Yongliang Shen 1 1 Zhejiang University 2 Apple 3 Tencent {zhengxilu, syl}@zju.edu.cn · 2026

Read UI-Copilot: Advancing Long-Horizon GUI Automation via Tool-Integrated Policy Optimization by Zhengxi Lu 1 , Fei Tang 1 , Guangyi Liu 1 , Kaitao Song 2 , Xu Tan 3 , Jin Ma 3 Wenqi Zhang 1 , Weiming Lu 1 , Jun Xiao 1 , Yueting Zhuang 1 , Yongliang Shen 1 1 Zhejiang University 2 Apple 3 Tencent {zhengxilu, syl}@zju.edu.cn (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-13822.txt.

Description

Lu and ten colleagues present UI-Copilot for long GUI tasks. Long tasks strain agents: memory fades, progress blurs, math hallucinates. UI-Copilot splits the work. The GUI agent executes; a lightweight copilot helps with memory retrieval and arithmetic. Memory decoupling separates observations from passing context. The agent calls the copilot as Retriever or Calculator. Tool-Integrated Policy Optimization trains tool choice and execution apart. The 7-billion-parameter model leads MemGUI-Bench and improves AndroidWorld by 17.1 percent over the base Qwen model.

Catalog record

CREATORZhengxi Lu 1 , Fei Tang 1 , Guangyi Liu 1 , Kaitao Song 2 , Xu Tan 3 , Jin Ma 3 Wenqi Zhang 1 , Weiming Lu 1 , Jun Xiao 1 , Yueting Zhuang 1 , Yongliang Shen 1 1 Zhejiang University 2 Apple 3 Tencent {zhengxilu, syl}@zju.edu.cn
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE57404
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.13822

Browse the full library · About the Tower of Babel