Mechanistically Guided LoRA Improves Paraphrase Consistency in Medical Vision-Language Models
Binesh Sadanandan, Vahid Behzadan · 2026
Read Mechanistically Guided LoRA Improves Paraphrase Consistency in Medical Vision-Language Models by Binesh Sadanandan, Vahid Behzadan (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-00148.txt.
Description
Sadanandan and Behzadan study why medical vision-language models flip answers when a clinical question is rephrased. On MIMIC-CXR binary questions, the baseline flips on about fifteen percent of pairs, with a sizeable gap. They train LoRA adapters with a joint loss balancing paraphrase consistency against answer accuracy, avoiding pure consistency training's mode collapse. The flip rate falls to about four percent and the margin gap shrinks by about four fifths, while accuracy holds near baseline; on PadChest the flip rate falls sharply as accuracy rises.
Catalog record
| CREATOR | Binesh Sadanandan, Vahid Behzadan |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 43175 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.00148