Don’t Show Pixels, Show Cues: Unlocking Visual Tool Reasoning in Language Models via Perception Programs

Muhammad Kamran Janjua Hugo Silva Di Niu Bahador Rashidi · 2026

Read Don’t Show Pixels, Show Cues: Unlocking Visual Tool Reasoning in Language Models via Perception Programs by Muhammad Kamran Janjua Hugo Silva Di Niu Bahador Rashidi (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-12896.txt.

Description

Muhammad Kamran Janjua and three colleagues rewrite vision tool outputs as language-native summaries. Dense pixel outputs clash with language reasoning strengths, so tools often fail to help. Perception Programs turn tool outputs into compact structured summaries models can parse. On six BLINK tasks, the method lifts base models and raw tool baselines. With GPT-5 Mini, multi-view reasoning jumps from 41.35 percent to 86.47 percent. Relative depth rises from about 52 percent to about 81 percent. Average gains reach 22 percent, with smaller models gaining 15 to 40 points.

Catalog record

CREATORMuhammad Kamran Janjua Hugo Silva Di Niu Bahador Rashidi
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE51287
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.12896

Browse the full library · About the Tower of Babel