To See is Not to Learn: Protecting Multimodal Data from Unauthorized Fine-Tuning of Large Vision-Language Model

Chengshuai Zhao, Zhen Tan, Dawei Li, Zhiyuan Yu, Huan Liu · 2026

Read To See is Not to Learn: Protecting Multimodal Data from Unauthorized Fine-Tuning of Large Vision-Language Model by Chengshuai Zhao, Zhen Tan, Dawei Li, Zhiyuan Yu, Huan Liu (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-14291.txt.

Introduction

Zhao and four co-authors propose MMGuard, proactive multimodal data protection against unauthorized fine-tuning of large vision-language models. Unauthorized scraping and training on multimodal web data pose IP and privacy risks to owners. MMGuard makes unlearnable samples by injecting imperceptible noise exploiting LVLM learning. The perturbation creates a shortcut by cutting training loss, causing overfitting to noise and degraded inference. A cross-modal binding disruption shifts LVLM attention to enforce a spurious noise-target link.

Catalog record

CREATORChengshuai Zhao, Zhen Tan, Dawei Li, Zhiyuan Yu, Huan Liu
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE140755
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.14291

Browse the full library · About the Tower of Babel