LEMON: Local Explanations via Modality-aware OptimizatioN

Yu Qin, Phillip Sloan, Raul Santos-Rodriguez, Majid Mirmehdi, Telmo de Menezes e Silva Filho · 2026

Read LEMON: Local Explanations via Modality-aware OptimizatioN by Yu Qin, Phillip Sloan, Raul Santos-Rodriguez, Majid Mirmehdi, Telmo de Menezes e Silva Filho (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-02786.txt.

Description

Yu Qin and 4 co-authors explain multimodal model predictions with one surrogate that respects each modality's distinct contribution, in a 2026 paper. Multimodal models are everywhere, but explanation methods tend to be single-modal, tied to one design, or too expensive to run at scale.

LEMON fits a single modality-aware surrogate model with group-structured sparsity around each prediction. The result separates how much each modality contributed from which features inside it mattered. The predictor is treated purely as a black box. Few forward passes are needed, and explanations stay faithful under repeated nudges. Testing spans vision-language question answering and a clinical task mixing image, text, and tabular inputs. Against strong multimodal rivals, faithfulness stayed competitive while black-box tests dropped by tens of times and runtime fell several-fold. Local explanation at fleet scale becomes workable.

Catalog record

CREATORYu Qin, Phillip Sloan, Raul Santos-Rodriguez, Majid Mirmehdi, Telmo de Menezes e Silva Filho
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE49 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602-02786

Browse the full library · About the Tower of Babel