Understanding Multimodal LLMs Under Distribution Shifts: An Information-Theoretic Approach
Changdae Oh, Zhen Fang, Shawn Im, Xuefeng Du, Yixuan Li · 2025
Read Understanding Multimodal LLMs Under Distribution Shifts: An Information-Theoretic Approach by Changdae Oh, Zhen Fang, Shawn Im, Xuefeng Du, Yixuan Li (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-00577.txt.
Description
Changdae Oh and co-authors build, in a 2025 theory paper, the first formal frame for measuring a multimodal model's risk under data shift. Such models do well in matched settings but stumble when test data differ from training data. That gap is a safety concern for real use. Past work mostly ran tests without a shared risk measure.
Their frame rests on a new metric that scores how well a reply fits its prompt, across image and text together. They bound how much that metric can drop when data shift. The drop is tied to gaps in each sense. Tests across sixty-one shift cases on real benchmarks matched the theory well, the authors report.
Catalog record
| CREATOR | Changdae Oh, Zhen Fang, Shawn Im, Xuefeng Du, Yixuan Li |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 105 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2502-00577