Omni-Safety under Cross-Modality Conflict: Vulnerabilities, Dynamics Mechanisms and Efficient Alignment
Kun Wang, Zherui Li, Zhenhong Zhou, Yitong Zhang, Yan Mi, Kun Yang, Yiming Zhang, Junhao Dong, Zhongxiang Sun, Qiankun Li, Yang Liu · 2026
Read Omni-Safety under Cross-Modality Conflict: Vulnerabilities, Dynamics Mechanisms and Efficient Alignment by Kun Wang, Zherui Li, Zhenhong Zhou, Yitong Zhang, Yan Mi, Kun Yang, Yiming Zhang, Junhao Dong, Zhongxiang Sun, Qiankun Li, Yang Liu (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-10161.txt.
Description
Omni-modal language models gain broad powers but also open cross-modal safety holes that single-modality testing misses. A 2026 paper led by Kun Wang, with ten co-authors, establishes a modality-semantics decoupling principle and builds the AdvBench-Omni data set, revealing a significant weak spot in omni-modal models. Mechanistic analysis finds a mid-layer fading effect tied to shrinking refusal-vector size, plus a mode-proof pure refusal direction.
Inspired by those findings, the team extracts a golden refusal vector by singular value decomposition and proposes OmniSteer, which uses lightweight adapters to tune intervention strength on the fly. Experiments lift refusal success against harmful inputs from about seventy percent to over ninety percent while keeping general powers across all modalities.
Catalog record
| CREATOR | Kun Wang, Zherui Li, Zhenhong Zhou, Yitong Zhang, Yan Mi, Kun Yang, Yiming Zhang, Junhao Dong, Zhongxiang Sun, Qiankun Li, Yang Liu |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 66 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2602-10161