Thinking with Images as Continuous Actions: Numerical Visual Chain-of-Thought
Kesen Zhao, Beier Zhu, Junbao Zhou, Xingyu Zhu, Zhongqi Yue, Hanwang Zhang · 2026
Read Thinking with Images as Continuous Actions: Numerical Visual Chain-of-Thought by Kesen Zhao, Beier Zhu, Junbao Zhou, Xingyu Zhu, Zhongqi Yue, Hanwang Zhang (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-23959.txt.
Description
Kesen Zhao and five co-authors present Numerical Visual Chain-of-Thought, a framework that lets multimodal language models reason over images with continuous coordinates. The 2026 paper notes that current visual reasoning grounds regions either as textified coordinates, which fragment meaning across modalities, or as fixed patches. Fixed patches limit precise selection and demand nontrivial architecture changes.
The new framework widens the model's action space from discrete tokens to continuous Euclidean space. In short, the model can point at places in a picture with smooth steps, using coordinates as actions. It supports both supervised fine-tuning and reinforcement learning, replacing categorical token policies with Gaussian or Laplace policies over coordinates and sampling through reparameterization for compatibility with group-relative policy optimization. Tests on three benchmarks against many baselines show better localization precision, answer accuracy, and faster training convergence.
Catalog record
| CREATOR | Kesen Zhao, Beier Zhu, Junbao Zhou, Xingyu Zhu, Zhongqi Yue, Hanwang Zhang |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 43 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2602-23959