Seeing and Reasoning with Confidence: Supercharging Multimodal LLMs with an Uncertainty-Aware Agentic Framework

Zhuo Zhi, Chen Feng, Adam Daneshmend, Mine Orlu, Andreas Demosthenous, Lu Yin, Da Li, Ziquan Liu, Miguel Rodrigues · 2025

Read Seeing and Reasoning with Confidence: Supercharging Multimodal LLMs with an Uncertainty-Aware Agentic Framework by Zhuo Zhi, Chen Feng, Adam Daneshmend, Mine Orlu, Andreas Demosthenous, Lu Yin, Da Li, Ziquan Liu, Miguel Rodrigues (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-08308.txt.

Description

Zhuo Zhi and co-authors give vision-language models a steadier way to reason in 2025. Such models answer questions about images, but multistep reasoning still trips them. Coaching them with written chains of thought is costly, while outside vision tools can inject fresh errors.

Their training-free frame lets the model pick image regions step by step, calling outside tools as needed. A statistical calibration step grades each tool's output for uncertainty, so the model leans on the most trustworthy one. Across five test sets, scores rose by about five points on average over the base model.

Catalog record

CREATORZhuo Zhi, Chen Feng, Adam Daneshmend, Mine Orlu, Andreas Demosthenous, Lu Yin, Da Li, Ziquan Liu, Miguel Rodrigues
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE46 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2503-08308

Browse the full library · About the Tower of Babel