ToolTok: Tool Tokenization for Efficient and Generalizable GUI Agents
Xiaoce Wang, Guibin Zhang, Junzhe Li, Jinzhe Tu, Chun Li, Ming Li · 2026
Read ToolTok: Tool Tokenization for Efficient and Generalizable GUI Agents by Xiaoce Wang, Guibin Zhang, Junzhe Li, Jinzhe Tu, Chun Li, Ming Li (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-02548.txt.
Description
Xiaoce Wang and 5 co-authors replace one-shot coordinate prediction with stepwise tool use for screen agents, in a 2026 arXiv paper. Agents that click by predicting coordinates generalize poorly across screen sizes and shapes. Coordinate-free options need more training data than exists.
ToolTok models screen operations as a sequence of progressive tool calls, in line with human interaction habits. Each tool is a learnable token embedding. An anchoring mechanism grounds every tool in related concepts, giving a helpful head start so embeddings can be learned from little supervision. A three-task curriculum, from token definitions through text-guided selection to simplified visual pathfinding, teaches a pretrained model the tool meanings step by step. The four-billion-parameter system beat models of like size and rivaled one nearly sixty times larger, using under one percent of rivals' training data.
Catalog record
| CREATOR | Xiaoce Wang, Guibin Zhang, Junzhe Li, Jinzhe Tu, Chun Li, Ming Li |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 54 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2602-02548