Expert-Data Alignment Governs Generation Quality in Decentralized Diffusion Models

Marcos Villagra, Bidhan Roy, Raihan Seraj, Zhiying Jiang · 2026

Read Expert-Data Alignment Governs Generation Quality in Decentralized Diffusion Models by Marcos Villagra, Bidhan Roy, Raihan Seraj, Zhiying Jiang (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-02685.txt.

Description

Marcos Villagra's 2026 study, joined by 3 co-authors, overturns the obvious guess about what makes decentralized diffusion models generate well. Such models route denoising through experts trained separately on disjoint data clusters. Stability seemed the natural governor: keep perturbations from amplifying during sampling and quality should follow.

Full ensemble routing, which averages every expert at each step, gives the most stable dynamics and best numerical convergence. It also produces by far the worst images. The governing principle is instead expert-data alignment: quality depends on routing each input to experts whose training data covers the current denoising state. Sparse top-two routing wins because it picks exactly those experts, as distance and per-expert accuracy analyses confirm. When experts disagree, quality degrades. Deployment guidance follows directly: optimize alignment, not stability metrics.

Catalog record

CREATORMarcos Villagra, Bidhan Roy, Raihan Seraj, Zhiying Jiang
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE60 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602-02685

Browse the full library · About the Tower of Babel