Rethinking Cross-Layer Information Routing in Diffusion Transformers
Chao Xu ∗,2 , Maohua Li ∗,1,2,♯ , Qirui Li 2,3,♯ , Yixuan Xu 2 , Yanke Zhou 1,2,♯ , Yunhe Li 2,4,♯ Cuifeng Shen 2 , Hanlin Tang †,2 , Kan Liu 2 , Tao Lan 2 , Lin Qu 2 , Shao-Qun Zhang §,1 · 2026
Read Rethinking Cross-Layer Information Routing in Diffusion Transformers by Chao Xu ∗,2 , Maohua Li ∗,1,2,♯ , Qirui Li 2,3,♯ , Yixuan Xu 2 , Yanke Zhou 1,2,♯ , Yunhe Li 2,4,♯ Cuifeng Shen 2 , Hanlin Tang †,2 , Kan Liu 2 , Tao Lan 2 , Lin Qu 2 , Shao-Qun Zhang §,1 (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-20708.txt.
Introduction
Chao Xu and eleven colleagues rethink how information flows across layers in diffusion transformers, the backbone of modern visual generation. The residual stream, which carries built-up information across layers, was inherited unchanged from the first transformer. Their analysis along depth and denoising time finds three symptoms: growing forward magnitudes, sharp backward gradient decay, and block-wise redundancy. They propose diffusion-adaptive routing, a drop-in replacement that aggregates past sublayer outputs in a learned, time-aware way.
Catalog record
| CREATOR | Chao Xu ∗,2 , Maohua Li ∗,1,2,♯ , Qirui Li 2,3,♯ , Yixuan Xu 2 , Yanke Zhou 1,2,♯ , Yunhe Li 2,4,♯ Cuifeng Shen 2 , Hanlin Tang †,2 , Kan Liu 2 , Tao Lan 2 , Lin Qu 2 , Shao-Qun Zhang §,1 |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 45525 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.20708
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Rethinking Cross-Layer Information Routing in Diffusion Transformers Chao Xu ∗,2 , Maohua Li ∗,1,2,♯ , Qirui Li 2,3,♯ , Yixuan Xu 2 , Yanke Zhou 1,2,♯ , Yunhe Li 2,4,♯ Cuifeng Shen 2 , Hanlin Tang †,2 , Kan Liu 2 , Tao Lan 2 , Lin Qu 2 , Shao-Qun Zhang §,1 Abstract Diffusion Transformers (DiTs) have become a de facto backbone of modern visual generation, and nearly every major axis of their design — tokenization, attention, conditioning, objectives, and latent autoencoders — has been extensively revisited. The residual stream that governs how information accumulates across layers, however, has been directly inherited from the original Transformer. In this paper, we present a systematic empirical analysis of cross-layer information flow in DiTs, jointly along depth and denoising timestep, and identify three concrete symptoms of traditional residual addition, namely monotonic forward magnitude inflation, sharp backward gradient decay, and pronounced block-wise redundancy. Motivated by this diagnosis, we propose Diffusion-Adaptive Routing ( DAR ), a drop-in residual replacement that performs learnable, timestep-adaptive, and non-incremental aggregation over the history of sublayer outputs. Moreover, the proposed DAR is compatible with many modern Transformer enhancement methods, such as REPA. On ImageNet 256 × 256 256\times 256 , DAR improves SiT-XL/2 by 2.11 2.11 FID ( 7.56 7.56 vs. 9.67 9.67 ) and matches the baseline’s converged quality with 8.75 × 8.75\times fewer training iterations. Stacked on top of REPA, it yields a 2 × 2\times training acceleration in the early stage, suggesting cross-layer information routing as an underexplored design axis in diffusion modeling, one that operates orthogonally to existing representation-alignment objectives. Beyond pretraining, DAR can also be applied during the fine-tuning stage of large-scale T2I models and preserves high-frequency details during Distribution Matching Distillation. \checkdata [E-mail][email protected], [email protected] 1 1 footnotetext: Equal contribution. Chao Xu initiated the project and built the codebase and infra; Maohua Li refined the idea, led the paper writing and REPA-related work. \dagger Project lead § Corresponding author ♯ Work done during internship at Alibaba (a) DMD visual comparison (b) Training FID curves Figure 1 : Overview of the main empirical results. (a) Our method preserves high-frequency details during DMD. (b) Training FID curves on ImageNet 256\times 256 . 1 Introduction Advances in the design and optimization of Diffusion Transformers (DiTs) that replace convolutional U-Nets with token-based Transformer denoisers [ 41 ] have led to significant breakthroughs in modern visual generation tasks [ 58 , 25 , 28 , 15 , 3 , 46 ] . A central challenge for modern visual generation with DiTs is to capture the time-varying dynamics of the denoising process by developing architectural innovations. Recent years have seen extensive efforts devoted to key components of DiTs, including macro structure design [ 2 , 41 , 13 , 31 ] , attention mechanisms [ 59 , 8 , 41 ] , conditioning mechanisms [ 52 , 68 ] , learning objectives [ 66 , 29 ] , latent autoencoders [ 63 , 9 , 69 ] , and causal and autoregressive DiTs [ 12 , 21 , 10 ] . However, the pre-normalized residual stream in DiTs and its variants — a fundamental design inherited from standard NLP practice — has remained largely unchanged, leaving open the question of its role in governing cross-layer information accumulation during the time-varying denoising process. This work starts with an in-depth investigation of cross-layer information routing in DiTs, jointly along depth and denoising timestep. On the one hand, our analysis suggests that this seemingly innocuous default residual addition in DiTs gives rise to three symptoms that emerge in lockstep with depth: hidden-state magnitudes inflate monotonically, backward gradients decay sharply, and adjacent transformer blocks become increasingly redundant, as shown in Fig. 2 . Strikingly, these symptoms collectively echo the PreNorm dilution phenomenon [ 61 ] recently characterized in Large Language Models (LLMs) [ 53 , 30 ] . On the other hand, cross-layer information flow within DiTs is inherently time-varying: as denoising progresses across a continuum of noise levels, the intermediate representations that matter most should shift from coarse-structure features in high-noise regimes to fine-detail features in low-noise regimes [ 18 , 45 ] . Thus, the fixed, time-agnostic, and uniform-weighted aggregation, as in conventional LLMs, is poorly suited to DiTs. Figure 2 : Three diagnostic symptoms of standard residual routing in DiTs across depth: forward magnitude inflation, backward gradient decay, and block-wise redundancy. Measured at t{=}1.0 . Several works have revisited the depth-wise structure of DiTs. A representative line of research [ 2 , 54 , 4 , 31 ] grafts U-Net-style long skip connections onto DiTs to bridge shallow and deep layers, with the goal of restoring the fixed hierarchical inductive bias of U-Nets, rather than enabling dynamic and timestep-aware aggregation across layers. Our key insight is that the denoising timestep — the very dimension that distinguishes DiTs from a standard Transformer — should play a vital role in adaptive routing. This motivates depth-wise aggregation mechanisms in DiTs to be learnable, timestep-adaptive, and non-incremental , so as to capture time-varying dynamics. Building on the above insights, this work elevates cross-layer information routing in DiTs from an inherited convention to an explicit design axis, with contributions on two complementary fronts. On the diagnostic side , we conduct, to the best of our knowledge, the first systematic study of cross-layer information flow in DiTs, decomposed jointly by depth and denoising timestep. We reveal that the three symptoms identified above and illustrated in Fig. 2 persist throughout training and vary systematically with the noise level, thereby suggesting that the role of the pre-normalized residual stream extends beyond stabilizing deep training, and exposing a spatiotemporal structure of PreNorm dilution that is invisible to LLM-side analyses. On the methodological side , we propose Diffusion-Adaptive Routing ( DAR ), a drop-in residual replacement that performs learnable, timestep-adaptive, and non-incremental aggregation. Inspired by [ 53 ] , we replace the running residual at each sublayer with a softmax attention over preceding sublayer outputs, where the query is computed from the current adaLN-modulated hidden state, allowing the routing mechanism to inherit both content and timestep dependence from DiT’s existing conditioning pathway. This preserves the isotropic and homogeneous Transformer stack without introducing manually specified layer pairing, and remains compatible with modern Transformer enhancement methods, such as REPA [ 66 ] . Empirically, on ImageNet 256 \times 256, DAR consistently outperforms vanilla SiT in our experiments, achieving 7.56 FID with SiT-XL/2 ( 2.11\downarrow over the baseline at matched compute) while matching the baseline’s converged quality in roughly 8.75\times fewer training iterations. Critically, the gains of DAR are orthogonal to representation-alignment objectives: combining DAR with REPA [ 66 ] yields a 2\times training acceleration in the early stage over REPA alone. This suggests that cross-layer information routing is a promising and underexplored direction for improving diffusion models, complementary to existing learning objectives. Quantitatively, the three dilution symptoms identified by our diagnosis tighten in lockstep with these FID gains, linking the diagnostic findings to the observed performance gains. Overall, the main contributions of this paper are summarized as follows • We conduct, to the best of our knowledge, the first comprehensive investigation of the cross-layer information flow in DiTs along both depth and denoising