LiteGUI: Distilling Compact GUI Agents with Reinforcement Learning
Yubin Wu Zicheng Cai Liping Ning Hua Wang · 2026
Read LiteGUI: Distilling Compact GUI Agents with Reinforcement Learning by Yubin Wu Zicheng Cai Liping Ning Hua Wang (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-07505.txt.
Introduction
Yubin Wu and six co-authors train lightweight agents that operate graphical interfaces through long, branching tasks. Small GUI agents struggle because several different actions can be valid at each state, and fixed demonstrations ignore that freedom. LiteGUI trains on logged interface states with teacher guidance that picks the best-matching valid action at each step.
A second stage mixes action-level supervision with planning-quality supervision, still respecting multiple valid actions. A scalable pipeline produced a matching multi-solution dataset for training and evaluation. Across GUI benchmarks, LiteGUI substantially improved small agents over standard fine-tuning and reinforcement approaches, rivaling much larger models.
Catalog record
| CREATOR | Yubin Wu Zicheng Cai Liping Ning Hua Wang |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 60 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.07505
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
LiteGUI: Distilling Compact GUI Agents with Reinforcement Learning Thanks: Corresponding author. [email protected] Yubin Wu Zicheng Cai Liping Ning Hua Wang Abstract Developing lightweight, on-device vision-language GUI agents is essential for efficient cross-platform automated interaction. However, current on-device agents are constrained by limited model capacity, and further performance improvements remain urgently needed. Traditional Supervised Fine-Tuning (SFT) for small-scale models often leads to overfitting, catastrophic forgetting and policy rigidity, and thus fails to fully address these challenges. In this work, we propose a novel SFT-free training paradigm that significantly enhances the performance of small-scale models. We first present the initial systematic integration of generalized knowledge distillation into the GUI agent domain via Guided On-policy Distillation . By incorporating oracle reference trajectories together with a dynamic retrieval mechanism, our method reduces hallucinations and mitigates the cognitive misalignment inherent in multi-solution GUI tasks. Building on this foundation, we further introduce a Multi-solution Dual-level GRPO framework that jointly aligns macro-level subtask planning with micro-level execution matching, thereby improving exploration in long-horizon GUI agent scenarios. In addition, we construct an automated data generation pipeline to synthesize GUI task trajectories with rich multi-solution annotations. Extensive experiments show that our method achieves state-of-the-art performance among lightweight models while remaining competitive with substantially larger-scale models across all benchmarks. Ablation studies further demonstrate that structured on-policy distillation and multi-solution dual-level exploration can fully unlock the capabilities of 2B–3B scale agents, surpassing the performance limits of conventional imitation learning. 1 Introduction Automated computer use has received increasing attention in recent years 17 ; 26 ; 4 ; 33 ; 5 ; 6 . Among these approaches, GUI agents represent a specialized class of systems that leverage visual understanding to interpret and execute computer-based tasks. While prior work has primarily focused on building GUI agents using large-scale models 27 ; 20 ; 2 , the development of lightweight models suitable for on-device or edge deployment remains largely underexplored. Smaller models are typically considered insufficient for handling such complex tasks due to their limited capacity. We observe that the commonly used supervised fine-tuning (SFT) paradigm—particularly for smaller models—tends to overfit to domain-specific and fixed interaction trajectories. This limitation degrades zero-shot generalization and hinders effective error recovery in dynamic UI environments. In this paper, we propose a new approach to improve the performance of lightweight GUI agents through on-policy distillation combined with reinforcement learning. While distillation has been successfully applied to tasks such as summarization, translation, and arithmetic reasoning (e.g., GKD 1 ; 8 ; 31 ), its effectiveness in GUI agent settings remains limited. This is largely due to the higher task complexity and the relatively weaker performance of available teacher models. To address the challenges, we propose Guided On-Policy Distillation (Guided-OPD) , which, to the best of our knowledge, represents the first attempt to apply distillation to the GUI agent domain. To ensure that the teacher model provides reliable guidance, our method incorporates prior knowledge by conditioning on reference ground-truth trajectories. Furthermore, to account for the multi-solution nature of GUI tasks, and to avoid over-reliance on a single reference trajectory that may be suboptimal and induce supervision–state mismatch, the proposed method dynamically infers the student’s exploration intent and retrieves the most-matched trajectory from a diverse solution pool in real time, providing adaptive guidance. This heuristic alignment mechanism remains compatible with single-solution scenarios while flexibly aligning with the student’s exploration solution, thereby significantly reducing the optimization difficulty for lightweight, on-device models. In addition, careful design of reinforcement learning (RL) strategies is essential to address the challenges inherent in GUI-based tasks. Existing RL approaches are often limited by their reliance on single-solution rewards, lack of long-horizon planning, and susceptibility to losing state context during execution. To overcome these limitations, we propose a dual-level reward framework that jointly supports long-term planning and short-term execution. At the macro level, we introduce robust, model-evaluated subtask rewards that provide structured guidance for decomposing complex, long-horizon tasks. At the micro level, by integrating real-time state perception and trajectory recording, the framework dynamically aligns low-level execution outcomes with an equivalent multi-solution set, thereby mitigating the “false negative” penalties induced by rigid single-label supervision. Experimental results demonstrate that this mechanism significantly improves execution precision while robustly preserving generalization across diverse and complex UI scenarios. Finally, to support these distillation and reinforcement learning exploration paradigms, we design an automated data generation pipeline and curate vision-only GUI trajectory datasets with extensive multi-solution annotations. Leveraging this pipeline, we further construct Lite-Dataset, comprising 30K GUI trajectory data and 11K annotated multi-solution samples across diverse computer-use scenarios. In addition, we introduce Lite-Bench, a new benchmark covering File system, Web and Terminal, with 160 samples, to systematically evaluate model performance in realistic GUI environments. Both Lite-Dataset and Lite-Bench will be publicly released to address the scarcity of high-quality data in the GUI agent domain. In summary, the core contributions of this paper are: New Training Paradigm. Motivated by the critical limitations of SFT —including catastrophic forgetting of foundational capabilities and overfitting to policy rigidity in small-scale models, we propose a novel SFT-free training paradigm of lightweight GUI agent models. By bypassing SFT entirely, our approach leverages OPD combined with RL to unlock the full potential of on-device 2B/A3B scale vision-only GUI agents; Guided On-policy Distillation. We incorporate OPD into the GUI agent domain and introduce oracle-guided references to reduce hallucinations in teacher supervision. Furthermore, for multi-step, multi-solution tasks, we propose a dynamic reference retrieval mechanism that mitigates cognitive misalignment while remaining fully compatible with single-solution scenarios; Multi-Solution Dual-Level GRPO. By integrating robust model-evaluated sub-task planning rewards with multi-solution action-set matching, we establish a dual-level RL framework for both long-term planning and short-term perception-execution. This design effectively alleviates the exploration bottlenecks inherent in long-horizon GUI tasks; Data Pipeline and Curated Dataset. We develop a data generation pipeline to semi-automatically produce multi-solution GUI task trajectories for training. With the pipeline we curate and open-source Lite-Dataset, a vision-only GUI dataset comprising 30K trajectories, including 11K high-quality multi-solution annotations, along with Lite-Bench, which contains 160 diverse evaluation instances; SOTA lightweight GUI-agent - LiteGUI. Evaluations on ScreenSpot-Pro 18 , OS-World 41 , and Lite-Bench demonstrate that our approach enables on-device models to achieve substantial gains in GUI task success rates. Our LiteGUI model achieves state-of-the-art performance among lightweight models and becomes competitive with much larger-scale models. Figure 1: Overview of Lite-GUI. The framework consists of: (a) an automated GUI trajectory generation module for producing action trajectories for GUI tasks; (b) an OPD & RL data construction module for generating training data, with a particular focus on multi-solution annotations for OPD and RL; and (c) the proposed two-stage training paradigm, comprising Guided On-policy Distillation and Multi-solution Dual-level GRPO. Details of the framework can refer to Appendix A.7 . 2 Related Work 2.1 Vision-based GUI Agents Early GUI agents predominantly relied on underlying system APIs or accessibility trees, which severely hindered their cross-platform generalization 10 ; 41 . Recently, the research paradigm has rapidly shifted toward vision-only agents, primarily bifurcating into two approaches: the direct application of powerful general-purpose Multimodal Large Language Models (MLLMs) (e.g., GPT-4o 17 , Claude 3.5 Sonnet, Qwen-VL 5 ; 6 ), and the fine-tuning of domain-specific GUI vision models (e.g., CogAgent 14 , UI-TARS 27 , OpenCUA 37 and