Omni-Safety under Cross-Modality Conflict: Vulnerabilities, Dynamics Mechanisms and Efficient Alignment

Kun Wang, Zherui Li, Zhenhong Zhou, Yitong Zhang, Yan Mi, Kun Yang, Yiming Zhang, Junhao Dong, Zhongxiang Sun, Qiankun Li, Yang Liu · 2026

Read Omni-Safety under Cross-Modality Conflict: Vulnerabilities, Dynamics Mechanisms and Efficient Alignment by Kun Wang, Zherui Li, Zhenhong Zhou, Yitong Zhang, Yan Mi, Kun Yang, Yiming Zhang, Junhao Dong, Zhongxiang Sun, Qiankun Li, Yang Liu (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-10161.txt.

Description

Omni-modal language models gain broad powers but also open cross-modal safety holes that single-modality testing misses. A 2026 paper led by Kun Wang, with ten co-authors, establishes a modality-semantics decoupling principle and builds the AdvBench-Omni data set, revealing a significant weak spot in omni-modal models. Mechanistic analysis finds a mid-layer fading effect tied to shrinking refusal-vector size, plus a mode-proof pure refusal direction.

Inspired by those findings, the team extracts a golden refusal vector by singular value decomposition and proposes OmniSteer, which uses lightweight adapters to tune intervention strength on the fly. Experiments lift refusal success against harmful inputs from about seventy percent to over ninety percent while keeping general powers across all modalities.

Catalog record

CREATORKun Wang, Zherui Li, Zhenhong Zhou, Yitong Zhang, Yan Mi, Kun Yang, Yiming Zhang, Junhao Dong, Zhongxiang Sun, Qiankun Li, Yang Liu
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE66 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602-10161

Browse the full library · About the Tower of Babel