What Semantics Survive the Connector? Diagnosing VLM-to-DiT Alignment in Video Editing

Hangyu Lin, Chao Wen, Chengming XuJianXiong Gao Jiangning ZhangXiaobin HuYanwei Fu HKUSTFDUZJUNUS · 2026

Read What Semantics Survive the Connector? Diagnosing VLM-to-DiT Alignment in Video Editing by Hangyu Lin, Chao Wen, Chengming XuJianXiong Gao Jiangning ZhangXiaobin HuYanwei Fu HKUSTFDUZJUNUS (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-20795.txt.

Description

Hangyu Lin and six colleagues diagnose what meaning lives on through the link between vision-language models and diffusion transformers in video editing. Flow-based video editors prepend vision-language models, on the view that a link can smoothly align rich multimodal reasoning with the text embedding space. The authors argue that link instead acts as a severe meaning bottleneck, harming fine-grained structural variables. A careful composition pipeline yields TRACE-Edit, a diagnostic set focused on relation-based editing, plus a protocol to separate link failures from generation errors.

Catalog record

CREATORHangyu Lin, Chao Wen, Chengming XuJianXiong Gao Jiangning ZhangXiaobin HuYanwei Fu HKUSTFDUZJUNUS
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE71576
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.20795

Browse the full library · About the Tower of Babel