PRISM: High-Resolution & Precise Counterfactual Medical Image Generation using Language-guided Stable Diffusion

Amar Kumar 1,2, Anita Kriz 1,2, Mohammad Havaei 3, Tal Arbel 1,2 · 2025

Read PRISM: High-Resolution & Precise Counterfactual Medical Image Generation using Language-guided Stable Diffusion by Amar Kumar 1,2, Anita Kriz 1,2, Mohammad Havaei 3, Tal Arbel 1,2 (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-00196.txt.

Description

Amar Kumar, Anita Kriz, Mohammad Havaei, and Tal Arbel (2025) present PRISM, a system that fabricates medical images with edited features. Deep learning for medical imaging is hampered by spurious correlations, imbalanced data, and scarce text labels: a model may learn to spot the scanner's marker instead of the disease. Counterfactual images, versions of a scan with a feature changed and the rest intact, can expose and correct shortcuts. PRISM makes them with language-guided Stable Diffusion.

Text prompts steer editing: a user can ask for a device removed or a disease sign added at high resolution. Control is precise: attributes change while unrelated anatomy stays put. Evaluation shows how the counterfactuals can audit models for reliance on spurious cues and balance skewed datasets. The work adapts vision-language foundation models, built for natural images, to such pictures.

Catalog record

CREATORAmar Kumar 1,2, Anita Kriz 1,2, Mohammad Havaei 3, Tal Arbel 1,2
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE31 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2503-00196

Browse the full library · About the Tower of Babel