Knowing When Not to Answer: Evaluating Abstention in Multimodal Reasoning Systems

Nishanth Madhusudhan, Vikas Yadav, Alexandre Lacoste · 2026

Read Knowing When Not to Answer: Evaluating Abstention in Multimodal Reasoning Systems by Nishanth Madhusudhan, Vikas Yadav, Alexandre Lacoste (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-14799.txt.

Description

Madhusudhan, Yadav, and Lacoste introduce MM-AQA, a benchmark for multimodal refusal. It builds unanswerable cases from answerable ones by varying modality needs and evidence supply. Tests covered 2,079 samples, three models, and two multi-agent designs. Under standard prompting, the models rarely abstained, beaten by simple confidence baselines. Multi-agent designs abstained more at an accuracy cost, and sequential designs matched iterative ones, with miscalibration the bottleneck. Models refused absent evidence but reconciled degraded or contradictory evidence, so training to abstain is needed, not more prompting or agents.

Catalog record

CREATORNishanth Madhusudhan, Vikas Yadav, Alexandre Lacoste
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE86217
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.14799

Browse the full library · About the Tower of Babel