Expert-Data Alignment Governs Generation Quality in Decentralized Diffusion Models Marcos Villagra, Bidhan Roy, Raihan Seraj, Zhiying Jiang Abstract Decentralized Diffusion Models (DDMs) route denoising through experts trained independently on disjoint data clusters, which can strongly disagree in their predictions. What governs the quality of generations in such systems? We present the first ever systematic investigation of this question. A priori, the expectation is that minimizing denoising trajectory sensitivity—minimizing how perturbations amplify during sampling—should govern generation quality. We demonstrate this hypothesis is incorrect: a stability–quality dissociation. Full ensemble routing, which combines all expert predictions at each step, achieves the most stable sampling dynamics and best numerical convergence while producing the worst generation quality (FID 47.9 vs. 22.6 for sparse Top-2 routing). Instead, we identify expert-data alignment as the governing principle: generation quality depends on routing inputs to experts whose training distribution covers the current denoising state. Across two distinct DDM systems, we validate expert-data alignment using (i) data-cluster distance analysis, confirming sparse routing selects experts with data clusters closest to the current denoising state, and (ii) per-expert analysis, showing selected experts produce more accurate predictions than non-selected ones, and (iii) expert disagreement analysis, showing quality degrades when experts disagree. For DDM deployment, our findings establish that routing should prioritize expert-data alignment over numerical stability metrics. 1 Introduction Decentralized Diffusion Models (DDMs) ( 17 ) combine independently trained diffusion experts ( 9 ) via an inference-time router. Because experts are trained on disjoint data clusters and can strongly disagree in their predictions, understanding what governs generation quality becomes crucial—yet this question has not been systematically studied. A natural hypothesis is that numerical stability determines quality ( 27 ) , that is, routing strategies that minimize trajectory sensitivity should produce superior samples. We demonstrate that this hypothesis is incorrect. In this work, we observe that full ensemble routing, in which all expert predictions are combined, achieves the lowest trajectory sensitivity and best numerical convergence, yet produces the worst generation quality. These results rule out trajectory sensitivity as the primary determinant of generation quality. We identify expert-data alignment (routing inputs to experts trained on similar data) as the governing principle. When sparse routing (e.g., Top-2) selects experts whose training distribution covers the current denoising state, each expert produces coherent velocity predictions that combine meaningfully. Full ensemble routing, by contrast, forces all experts to process every input; since each expert is trained on only a subset of the data, most of them are processing out-of-distribution data at any given time. The averaged velocity field may be smooth, but it points toward an incoherent compromise rather than the data manifold. We provide direct experimental validation of this principle across two distinct DDM systems. Data-cluster distance analysis confirms that sparse routing selects experts with data clusters closest to the input embedding. Per-expert prediction quality analysis shows that selected experts produce velocity predictions with higher alignment to the blended output. Expert disagreement analysis demonstrates that disagreement under full ensemble correlates with quality degradation. Although numerical stability does not govern quality, understanding when DDM sampling converges remains valuable. Classical stability analysis suggests DDMs should fail: Lipschitz constants of deep networks grow exponentially with depth ( 4 ; 26 ; 27 ) , and Grönwall’s inequality implies small perturbations amplify over integration ( 7 ) . We also examine trajectory-local sensitivity , denoted \widehat{L}_{\text{eff}}^{(h)} , which formalizes the idea that Jacobian spectral norms remain bounded along realized sampling paths. Empirically, we observe that trajectories exhibit moderate sensitivity compared to worst-case global bounds. While \widehat{L}_{\text{eff}}^{(h)} does not predict quality across routing strategies, it may serve as a within-strategy diagnostic for identifying numerically sensitive samples. The contributions of this work are as follows. • Expert-data alignment principle. We identify expert-data alignment as the primary determinant of generation quality in DDMs. We provide direct experimental validation via (i) data-cluster distance analysis showing sparse routing selects experts with data clusters closest to the input, (ii) per-expert analysis demonstrating that selected experts produce superior velocity predictions, and (iii) expert disagreement analysis showing quality degrades when experts disagree. • Stability–quality dissociation. We demonstrate that trajectory sensitivity does not govern generation quality: full ensemble routing achieves the lowest \widehat{L}_{\text{eff}}^{(h)} and step-refinement disagreement, yet produces the worst Fréchet Inception Distance (FID) ( 8 ) . This rules out numerical stability as the primary quality determinant. We additionally explore trajectory-local sensitivity as a within-strategy diagnostic, finding weak predictive power. This paper is organized as follows. Section 2 reviews related work. Section 3 introduces notation and background on DDMs. Section 4 establishes the stability–quality dissociation. Section 5 identifies expert-data alignment as the governing principle and provides direct experimental validation. Section 6 presents trajectory sensitivity analysis and empirical validation. Sections 7 and 8 discuss implications and conclude. 2 Related Work Diffusion as ODE/SDE and numerical stability. Diffusion sampling involves Lipschitz-constrained ODEs where discretization error affects accuracy. Sampling can be expressed as a probability-flow Ordinary Differential Equation (ODE) ( 23 ) , where Lipschitz constants and discretization error determine solver accuracy. Recent work by 25 addresses temporal stiffness via a specialized solver called STORK based on stabilized Runge-Kutta methods for stiff diffusion ODEs. STORK’s stabilized solvers could, in principle, be combined with our approach (using stable solvers for temporal stiffness while using sparse routing to control spatial sensitivity). We use the released Euler and Heun solvers to isolate the effect of routing strategies, but STORK-style solvers may provide additional gains in decentralized settings. Local Lipschitz analysis. Local Lipschitz bounds have been extensively studied in the neural network literature. 12 developed methods for exactly computing local Lipschitz constants, enabling input-specific sensitivity analysis rather than global worst-case bounds. Our work applies this trajectory-local perspective to diffusion sampling. The key novelty is not the local Lipschitz concept itself, but its application to understanding decentralized diffusion dynamics and the discovery that routing implicitly stabilizes the sampling dynamics. Decentralized diffusion. DDMs show that decentralized experts, when routed, can match a monolithic diffusion objective ( 17 ) . Our work investigates why such combination succeeds despite expert disagreement, providing a stability-based explanation complementary to the original capacity arguments. DDM vs. traditional Mixture-of-Experts. DDMs are ensembles of independently trained models , not traditional Mixture-of-Experts (MoE). In standard MoE architectures experts are FFN layers within a shared backbone, trained jointly with load balancing losses, and routed at the token level ( 21 ; 5 ) . In DDM, each “expert” is a complete diffusion model trained in isolation on a disjoint data partition (no shared parameters, no gradient communication, no joint training). Routing occurs at the input level (entire noisy images) rather than token level, and experts are combined only at inference time. Concurrent work applies traditional MoE architectures within diffusion models ( 6 ; 24 ; 22 ; 28 ; 1 ; 16 ; 30 ) ; DDM instead combines complete, independently trained models. This distinction matters for stability analysis: DDM experts can produce arbitrarily different outputs for the same input (having never coordinated during training), whereas MoE experts share a representational backbone that constrains their disagreement. Our analysis specifically addresses the stability challenges arising from DDM’s decentralized training. Data-aware routing and expert specialization. The importance of matching inputs to appropriately trained experts is well-established in MoE systems. Sparsely-gated MoE architectures rely on learned routing to direct inputs to relevant experts ( 21 ) , with subsequent work analyzing expert utilization and load balancing ( 3 ; 31 ) . In federated learning, data heterogeneity across clients creates analogous challenges: models trained on non-IID partitions may produce poor predictions on out-of-distribution inputs ( 14 ) . Our work provides direct experimental evidence that this principle governs sample quality in DDMs: sparse routing succeeds precisely because it maintains alignment between inputs and expert training distributions. 3 Background 3.1 Notation For a differentiable scalar function f:\mathbb{R}^{d}\times[0,1]\to\mathbb{R} , we let \nabla_{x}f(x,t)\in\mathbb{R}^{d} denote the gradient with respect to x . We use J_{x}f(x,t)\in\mathbb{R}^{d\times d} to denote the Jacobian matrix. We use \|\cdot\| for the matrix spectral norm and to denote the \ell_{2} -norm over \mathbb{R}^{d} . 3.2 Diffusion sampling Diffusion sampling generates data by integrating an ordinary differential equation (ODE) \frac{dx_{t}}{dt}=v_{t}(x_{t}) from noise to data ( 15 ) . We call v=\{v_{t}\}_{t=0}^{1} a flow and the solution \{x_{t}\} a trajectory . The ODE has a unique solution when v is Lipschitz continuous ( 2 ) . 3.3 Decentralized Diffusion Decentralized Diffusion Models (DDMs) ( 17 ) train K expert diffusion models in complete isolation on disjoint data partitions. Unlike traditional MoE (which routes tokens to different FFN layers within a shared backbone), DDM routes entire inputs to separate full models at each denoising step. At inference, a lightweight router predicts weights w_{t}^{(k)}(x_{t})\geq 0 with \sum_{k=1}^{K}w_{t}^{(k)}(x_{t})=1 at each denoising step. The routed velocity field is given by v_{t}(x_{t})=\sum_{k=1}^{K}w_{t}^{(k)}(x_{t})\,v_{t}^{(k)}(x_{t}). (1) Sampling integrates \frac{dx_{t}}{dt}=v_{t}(x_{t}) from noise x_{0}\sim\mathcal{N}(0,I) . The router outputs probabilities p_{t}(k|x_{t}) for each expert k . Full ensemble mode uses w_{t}^{(k)}(x_{t})=p_{t}(k|x_{t}) for all k . Top- k routing selects the k experts with highest probability and renormalizes their weights ( 21 ) . Top-1 selects a single expert: v_{\text{Top-1}}(x_{t})=v_{k^{*}}(x_{t}) with k^{*}=\arg\max_{k}p_{t}(k|x_{t}) . This yields a piecewise-smooth vector field with non-differentiability on measure-zero switching surfaces. In this work, when we say that the DDM router converges we mean numerical convergence of the sampler, formalized below. Definition 3.1 (DDM sampling convergence) . Fix trained experts and a router with a routed flow v=\{v_{t}\}_{t=0}^{1} . Let x_{1}\sim\mathcal{N}(0,I) be an initial condition at time t=1 and let x_{t} denote the ODE solution at time t<1 (note that t is decreasing). Let \tilde{x}^{(h)}_{t} denote the output of a numerical ODE solver with step size h , coupled to the same initial noise x_{1} . We say the DDM sampler converges in probability if for every \varepsilon>0 , we have P\big(\|\tilde{x}^{(h)}_{0}-x_{0}\|>\varepsilon\big)\to 0\quad\text{as }h\to 0, where the probability is over the initial noise x_{1} . This notion implies convergence in distribution of \tilde{x}^{(h)}_{0} to x_{0} . 4 The Stability–Quality Dissociation A natural hypothesis is that numerical stability governs generation quality in DDMs, that is, routing strategies that minimize trajectory sensitivity should produce superior samples. We establish that this hypothesis is incorrect. Sections 5 and 6 will demonstrate the dissociation: full ensemble routing achieves the lowest trajectory sensitivity and lowest step-refinement disagreement, yet produces the worst generation quality—even worse generation quality is observed in ( 11 ) . Top-2 achieves the best generation quality despite higher trajectory sensitivity. This rules out numerical stability as the quality determinant. 5 Expert-Data Alignment The preceding section ruled out numerical stability as the quality determinant. We now establish expert-data alignment —routing inputs to experts trained on similar data—as the governing principle and provide direct experimental validation. 5.1 The Expert-Data Alignment Hypothesis Full ensemble averaging produces a smoother velocity field because: (1) averaging over all experts reduces variance in velocity predictions, (2) this variance reduction directly lowers \|J_{x}v(x_{t},t)\| , and (3) smoother trajectories also exhibit lower step-refinement disagreement. However, this smoothing forces experts to process out-of-distribution data. Each expert is trained on only small part of the data (one cluster); when all experts contribute to every input, most of them process data outside their training distribution. Therefore, the averaged velocity field may be smooth but it points toward an incoherent compromise rather than the data manifold. Top-2 routing selects the two experts whose training data most closely matches the current input, keeping each expert producing coherent velocity predictions that combine meaningfully. The dominant factor for sample quality is whether experts process data similar to their training distribution (what we call expert-data alignment ) rather than minimizing trajectory sensitivity. If expert-data alignment governs quality, we expect that 1. sparse routing should achieve higher alignment (selected experts have lower cluster distance) than full ensemble; 2. selected experts should produce superior velocity predictions compared to non-selected experts; and, 3. expert disagreement should correlate with quality degradation under full ensemble. 5.2 Cluster Distance Analysis We use the pretrained DDM Paris model ( 11 ) via released pretrained checkpoints. The model consists of K=8 experts trained on a subset of LAION-Aesthetics ( 20 ) . The dataset was partitioned into 8 semantic clusters via two-stage hierarchical k-means on DINOv2-ViT-L/14 embeddings. The model uses a DiT-B/2 router ( \sim 129M parameters) and 8 DiT-XL/2 experts (a modified version of the DiT-XL/2 experts with \sim 606M parameters each, \sim 5B total). The router was trained post-hoc on the full dataset, effectively learning to route inputs to the expert trained on the most similar data. We will use Paris DDM to test the Expert-Data Alignment Hypothesis. Let \mathcal{C}_{k} denote the training data cluster for expert k , and let d(x,\mathcal{C}_{k}) denote the Euclidean distance from the DINOv2 embedding of input x to the centroid of cluster \mathcal{C}_{k} . We define high expert-data alignment as the condition where the selected experts have low d(x,\mathcal{C}_{k}) relative to non-selected experts. We test whether sparse routing selects experts whose training clusters match the input distribution. For n=500 samples, we extract DINOv2-ViT-L/14 embeddings at timesteps t\in\{0.3,0.5,0.7\} during sampling. For each state (x_{t},t) , we compute: (1) the Euclidean distance from the embedding to each of the 8 cluster centroids used during expert training; (2) the experts selected by each routing strategy at that timestep; (3) the rank of the selected expert(s)’ cluster(s) among all 8 clusters, ordered by distance. We emphasize that this embedding distance is used only for relative expert ranking; all conclusions depend solely on rank comparisons, not metric fidelity. Moreover, the training clusters themselves were defined via k-means in DINOv2 space, making this the canonical embedding for cluster proximity. We report two metrics. (i) the average rank (1 = closest, 8 = farthest) of the selected expert’s training cluster refered to as Mean cluster rank , and (ii) the percentage of timesteps where at least one selected expert’s cluster is among the two closest to the current input refered to as Top-2 Match Rate . Table 1 shows the results. Table 1 : Cluster distance analysis validates expert-data alignment. Sparse routing selects experts whose training clusters match the input. Lower mean rank indicates better alignment (rank 1 = closest). Results averaged over n=500 samples at t\in\{0.3,0.5,0.7\} . We also include the results for full ensemble as a baseline reference. Routing Mean Cluster Rank \downarrow Top-2 Match Rate \uparrow Top-1 1.54 \pm 0.28 90.2% Top-2 1.96 \pm 0.26 83.9% Full (8) 4.50 \pm 0.00 25.0% Top-1 and Top-2 achieve mean cluster ranks of 1.54 and 1.96, far below the 4.5 random baseline, with Top-2 match rates exceeding 83%. 5.3 Per-Expert Prediction Quality Now we want to test whether selected experts produce superior velocity predictions compared to non-selected experts. For n=200 samples generated with Top-2 routing, we record at each timestep (1) the blended velocity v_{t}(x_{t})=\sum_{k\in\mathcal{S}}w_{t}^{(k)}v_{t}^{(k)}(x_{t}) , (2) the individual velocity predictions v_{t}^{(k)}(x_{t}) for all 8 experts, and (3) the routing weights and selected set \mathcal{S} . For each expert k at each timestep, we compute the velocity alignment score define as a^{(k)}(x_{t})=\frac{v_{t}^{(k)}(x_{t})^{\top}\cdot v_{t}(x_{t})}{\|v_{t}^{(k)}(x_{t})\|\cdot\|v_{t}(x_{t})\|}, where v_{t}^{(k)}(x_{t})^{\top} denotes the transpose of v_{t}^{(k)}(x_{t}) . This cosine similarity measures how well expert k ’s prediction aligns with the routed velocity used for successful generation. We report angular deviation \theta_{k}=\arccos(a_{k}) in degrees for interpretability. See Table 2 for the results. Table 2 : Selected experts produce better-aligned predictions. Angular deviation (degrees) from blended velocity for selected versus non-selected experts under Top-2 routing. Smaller is better. Results averaged over n=200 samples. Statistical significance via independent samples t-test. Expert Status Angular Dev. \downarrow Std Dev Selected (Top-2) 3.6° \pm 1° Non-selected 5.1° \pm 1° Reduction 1.5° (29%) p<0.001 Selected experts achieve smaller angular deviation from the blended velocity (3.6° vs. 5.1°; independent samples t-test, p<0.001 ), a 29% reduction confirming systematic identification of coherent experts. The statistical significance demonstrates that routing systematically identifies the most coherent experts rather than selecting arbitrarily. 5.4 Expert Disagreement and Sample Quality In this section, we want to test whether expert disagreement explains the poor quality of full ensemble routing. For n=500 samples generated with full ensemble, we measure (1) mean pairwise expert disagreement D(x_{t})=\frac{1}{\binom{K}{2}}\sum_{i0 depending on the solver. Conditioning on this event and choosing h sufficiently small yields, for any \varepsilon>0 , P\big(e_{N}>\varepsilon\big)\leq P\big(L_{\text{eff}}(x_{1})>L\big)\leq\delta. Full convergence in probability ( \lim_{h\to 0}P(e_{N}>\varepsilon)=0 for all \varepsilon>0 ) follows by taking L\to\infty along quantiles of L_{\text{eff}} and adjusting h accordingly; see Appendices B for details. Definitions 6.1 – 6.4 are stated in terms of the exact ODE solution \{x_{t}\} , which is not directly accessible in practice. Accordingly, our experiments report \widehat{L}_{\text{eff}}^{(h)} (Definition 6.2 ) computed along numerical trajectories as a proxy. This is evidence for sensitivity and it is not, by itself, a guarantee that L_{\text{eff}} is bounded for the exact flow. All convergence statements below are therefore conditional on the (unobserved) typical-set of trajectories whose effective Lipschitz constant stays bounded. 6.2 Empirical Validation of the Convergence Argument We design three experiments to validate the convergence argument, each targeting one step of the logical chain: (1) bounded local error, (2) bounded error propagation via trajectory-local sensitivity, and (3) convergence under step-size refinement. Appendices C and D provide additional technical details. 6.2.1 Experiment 1: Verifying Bounded Local Discretization Error The goal of this experiment is to test that the numerical solver introduces bounded local truncation error at each step. We measure local truncation error by comparing single-step outputs between Heun’s method and a high-precision reference (Heun with h/10 subdivisions). We compute this at 1000 randomly selected trajectory points across 1,000 samples for each routing strategy. The following measurements are computed: 1) mean and maximum local error \epsilon^{\text{local}} per routing strategy, 2) scaling with step size at h\in\{0.02,0.01,0.005\} for each routing strategy, and 3) distribution of local errors across timesteps and samples for each routing strategy. Detailed results are reported in Appendix D.2 . 6.2.2 Experiment 2: Measuring Trajectory-Local Sensitivity The goal of this experiment is to test that the trajectory exhibits bounded Jacobian spectral norms, and that this bounds error propagation. We emphasize that spectral norms quantify worst-case local sensitivity; we do not estimate Jacobian eigenvalues or logarithmic norms along trajectories, and therefore do not directly characterize linear stability of the flow. We treat this as a limitation and leave eigenvalue-based analyses to future work. For each numerical sampling trajectory \{\tilde{x}^{(h)}_{t_{n}}\}_{n=0}^{N} starting from x_{1} we compute \widehat{L}_{\text{eff}}^{(h)}(x_{1}) . However, computing a Jacobian spectral norm can be time consuming for latent spaces of high dimension, and hence, we estimate it using the power method. We repeat the power iteration until the relative change between iterations 9 and 10 is less than 0.5\% . For samples with slower convergence, we extend up to 20 iterations. The following measurements are computed: 1) \widehat{L}_{\text{eff}}^{(h)} distribution per routing strategy including mean and standard deviation, and 2) temporal profile of \|J_{x}v(\tilde{x}^{(h)}_{t_{n}},t_{n})\|_{2} as a function of t_{n} . Results are reported in Appendix D.3 and Figure 2 . Figure 2 : Cumulative IQR of the Jacobian spectral norm \|\nabla_{x}v(x_{t},t)\| as a measure of variability across sampling trajectories. The gap between Top-2 and other strategies widens as denoising progresses. Mid-trajectory timesteps ( t\in[0.1,0.9] ); n{=}20 samples per strategy. Table 4 : Routing strategy comparison. FID from ( 11 ) ; \widehat{L}_{\text{eff}}^{(h)} and \Delta_{\text{refine}} measured on n{=}1000 samples. Full ensemble achieves the lowest trajectory sensitivity and step-refinement disagreement yet produces the worst FID. Strategy FID \downarrow \boldsymbol{\widehat{L}_{\textbf{eff}}^{(h)}} \boldsymbol{\Delta_{\textbf{refine}}} \downarrow Monolithic 29.64 – – Top-1 30.60 18.81 \pm 7.15 0.075 \pm 0.107 Top-2 22.60 17.48 \pm 6.07 0.051 \pm 0.070 Full (8) 47.89 17.07 \pm 6.33 0.020 \pm 0.020 6.2.3 Experiment 3: Step-Size Refinement and Convergence The goal of this experiment is to test whether step-size refinement reduces endpoint error, and whether step-refinement disagreement relates to trajectory-local stability. For each initial noise x_{1} , we generate samples at N steps (step size h=1/N ) and 2N steps (step size h/2 ). We measure the step-refinement disagreement defined as \Delta_{\text{refine}}(x_{1}):=\text{LPIPS}\big(D(\tilde{x}_{0}^{(h)}),D(\tilde{x}_{0}^{(h/2)})\big), (2) where D(\cdot) denotes VAE decoding, LPIPS measures perceptual distance in decoded image space ( 29 ) , and x_{1} is the initial noise. The metric \Delta_{\text{refine}} is an a posteriori refinement-based error indicator that does not involve Jacobian computation, providing a measure of numerical integration error that avoids methodological coupling with L_{\text{eff}} . The following measurements are computed: 1) \Delta_{\text{refine}} distribution per routing strategy, 2) Spearman correlation coefficient \rho(\widehat{L}_{\text{eff}}^{(h)},\Delta_{\text{refine}}) , and 3) area under the curve (AUC) for predicting high- \Delta_{\text{refine}} events using \widehat{L}_{\text{eff}}^{(h)} to measure predictive performance. Results are reported in Figure 3 ; see Appendix D.4 for methodology details. 6.3 Summary of Results Table 4 shows a summary of the observed results. Across all routing strategies, correlation between \widehat{L}_{\text{eff}}^{(h)} and step-refinement disagreement \Delta_{\text{refine}} is low ( \rho<0.08 ; Figure 3 ) 1 1 1 Preliminary analysis of the router Jacobian term |\sum_{k}v_{k}\nabla_{x}w_{k}| also shows similarly low correlation with \Delta_{\text{refine}} (Spearman \rho=-0.07 , p=0.62 for Top-2 routing, n{=}50 ), reinforcing that Jacobian-based sensitivity metrics are not aligned with discretization error. . This further supports our finding that numerical stability metrics do not govern generation quality, aligning with the observed dissociation between sensitivity and FID. Factors beyond worst-case Jacobian norms—such as directional alignment of perturbations with the flow or cancellation effects across timesteps—likely govern actual error accumulation. Figure 3 : Correlation between trajectory sensitivity and step-refinement disagreement from Experiment 3 ( n{=}1000 samples per routing strategy). (a) Spearman correlation \rho(\widehat{L}_{\text{eff}}^{(h)},\Delta_{\text{refine}}) is weak across all strategies ( \rho<0.08 ), indicating that L_{\text{eff}} is not a tight predictor of discretization error. (b) Mean step-refinement disagreement \Delta_{\text{refine}} shows clear ordering: Full ensemble achieves the lowest discretization error (0.020), followed by Top-2 (0.051) and Top-1 (0.075). 7 Discussion The experimental results reveal a fundamental tradeoff: strategies that maximize numerical stability (full ensemble) necessarily sacrifice expert-data alignment, and vice versa. Cluster distance analysis quantifies this directly—sparse routing achieves mean cluster ranks of 1.54–1.96 versus 4.50 for full ensemble (Table 1 )—while per-expert analysis shows selected experts produce 29% lower angular deviation from the blended velocity (Table 2 ). The disagreement-quality correlation provides shows that when alignment breaks down, then sample quality degrades proportionally. Furthermore, we saw that L_{\text{eff}} is not a cross-strategy quality predictor, but remains useful for within-strategy diagnostics to identify numerically sensitive trajectories. The weak correlation \rho(L_{\text{eff}},\Delta_{\text{refine}})\approx 0.03 – 0.08 suggests factors beyond worst-case Jacobian norms affect actual error accumulation. The step-refinement ordering confirms that smoother velocity fields yield better numerical convergence, yet this convergence advantage does not translate to better generation quality. 8 Conclusions We investigated what governs generation quality in Decentralized Diffusion Models (DDMs), where independently trained experts are combined via inference-time routing. Our central finding is that expert-data alignment governs generation quality : routing inputs to experts trained on similar data is the primary determinant of quality, not numerical stability. We provide direct experimental validation through cluster distance analysis (showing sparse routing selects in-distribution experts), per-expert prediction quality (showing selected experts produce superior velocities), and disagreement analysis (explaining why full ensemble fails). For practitioners, our findings demonstrate that when deploying DDMs with independently trained experts, routing that maintains expert-data alignment is more important than optimizing for numerical stability metrics. Future work should explore training objectives that improve expert robustness to out-of-distribution inputs. In Appendix E we discuss limitations and future directions. Broader Impact This work analyzes routing strategies in Decentralized Diffusion Models, with implications for computational efficiency and scientific understanding of expert ensemble systems. Computational Impact. Our findings demonstrate that sparse routing strategies (e.g., Top-2) achieve superior generation quality compared to full ensemble averaging while requiring 4 \times fewer active experts at inference time. This efficiency gain may reduce computational costs and energy consumption in deployed generative systems. Exploratory Analysis. We examined trajectory sensitivity ( \widehat{L}_{\text{eff}}^{(h)} ) as a potential diagnostic tool. While correlation with step-refinement error was weak, the analysis methodology may inform future work on identifying numerically sensitive samples in decentralized expert architectures. Limitations. Our analysis uses pretrained models on LAION-Aesthetics, which contains known demographic biases. We perform inference-only analysis without introducing new generative capabilities. No large-scale training is conducted. Appendix Appendix A When \widehat{L}_{\text{eff}}^{(h)} approximates L_{\text{eff}} Assume the exact ODE solution \{x_{t}\}_{t\in[0,1]} exists and remains in a set K , and that for all t\in[0,1] , J_{x}v(\cdot,t) is L_{J} -Lipschitz on K , that is, \|J_{x}v(x,t)-J_{x}v(y,t)\|\leq L_{J}\|x-y\| for all x,y\in K . Let \tilde{x}^{(h)}(\cdot) be any continuous-time interpolation of the numerical trajectory such that \sup_{t\in[0,1]}\|x_{t}-\tilde{x}^{(h)}(t)\|\leq\eta(h) with \eta(h)\to 0 as h\to 0 , and define \widehat{L}_{\text{eff,cont}}^{(h)}(x_{1}):=\sup_{t\in[0,1]}\|J_{x}v(\tilde{x}^{(h)}(t),t)\|. Then \big|L_{\text{eff}}(x_{1})-\widehat{L}_{\text{eff,cont}}^{(h)}(x_{1})\big|\leq L_{J}\,\eta(h), so \widehat{L}_{\text{eff,cont}}^{(h)}(x_{1})\to L_{\text{eff}}(x_{1}) as h\to 0 . Moreover, if t\mapsto\|J_{x}v(\tilde{x}^{(h)}(t),t)\| is L_{t} -Lipschitz on [0,1] , then the grid maximum in Definition 6.2 satisfies 0\leq\widehat{L}_{\text{eff,cont}}^{(h)}(x_{1})-\widehat{L}_{\text{eff}}^{(h)}(x_{1})\leq L_{t}h, so \widehat{L}_{\text{eff}}^{(h)}(x_{1}) is a consistent proxy for L_{\text{eff}}(x_{1}) under refinement. This shows that the gap between this grid maximum and a continuous-time supremum is O(h) . Appendix B Probabilistic Convergence under Empirical Stability This appendix provides the formal convergence argument sketched in Section 6.1 . We use a standard conditioning approach combined with deterministic ODE error bounds. Proposition B.1 (Conditional convergence) . Let v be a velocity field and q a noise distribution. Suppose that for x_{1}\sim q , the effective Lipschitz constant L_{\emph{eff}}(x_{1}) (Definition 6.1 ) satisfies P(L_{\emph{eff}}(x_{1})<\infty)=1 . Let \tilde{x}^{(h)}_{t_{N}} denote the numerical solution at the final step using step size h , and let e_{N}:=\|x_{t_{N}}-\tilde{x}^{(h)}_{t_{N}}\| be the global error. Then for any \varepsilon>0 , \lim_{h\to 0}P(e_{N}>\varepsilon)=0. Proof. Fix \varepsilon>0 and \eta>0 . Define the events A_{L}:=\{L_{\text{eff}}(x_{1})\leq L\} and E:=\{e_{N}>\varepsilon\} . By the law of total probability, P(E)\leq P(E\mid A_{L})+P(A_{L}^{c}). Step 1 (Deterministic bound on A_{L} ). Standard Grönwall-based global error analysis for one-step methods (see, e.g., 7 ) shows that if the velocity field has Lipschitz constant L along the trajectory, then the global error satisfies e_{N}\leq Ch^{p}e^{L} for constants C>0 and p\geq 1 depending on the solver order and local truncation error bounds. On the event A_{L} , the trajectory-local Lipschitz constant is bounded by L , so this deterministic bound applies. Choosing h^{*}=h^{*}(\varepsilon,L):=(\varepsilon/(Ce^{L}))^{1/p} ensures e_{N}\leq\varepsilon on A_{L} for all h\leq h^{*} . Thus P(E\mid A_{L})=0 for h\leq h^{*} . Step 2 (Choosing L ). Since P(L_{\text{eff}}(x_{1})<\infty)=1 , for any \eta>0 there exists L=L(\eta) such that P(A_{L}^{c})=P(L_{\text{eff}}(x_{1})>L)<\eta . Step 3 (Two-parameter limit). Given \varepsilon,\eta>0 : (i) choose L=L(\eta) so that P(A_{L}^{c})<\eta ; (ii) choose h\leq h^{*}(\varepsilon,L) so that P(E\mid A_{L})=0 . Then P(E)<\eta . Since \eta>0 was arbitrary, \lim_{h\to 0}P(e_{N}>\varepsilon)=0 . ∎ Remark B.2 . The key assumption is P(L_{\text{eff}}(x_{1})<\infty)=1 , i.e., that almost all trajectories have finite effective Lipschitz constant. This is an empirical regularity condition that we validate by measuring \widehat{L}_{\text{eff}}^{(h)} along numerical trajectories. The (L,\delta) -trajectory-locally sensitive condition (Definition 6.4 ) connects directly to this result: since the definition requires the bound to hold for some L and \delta , we are free to choose any \delta>0 and set L=L(\delta) as the corresponding quantile of L_{\text{eff}} . For this (L(\delta),\delta) pair and sufficiently small h=h(\varepsilon,L(\delta)) , we obtain P(e_{N}>\varepsilon)\leq\delta . Since \delta can be made arbitrarily small, this recovers full convergence in probability. Appendix C Expert-Data Alignment Experiment Details This section provides implementation details for the alignment experiments described in Section 5 . DINOv2 embedding extraction. We use DINOv2-ViT-L/14 ( 18 ) to extract embeddings for both training data cluster centroids and intermediate sampling states. For intermediate states x_{t} during sampling, we first decode through the VAE to obtain pixel-space images, then extract DINOv2 embeddings. The 8 cluster centroids were computed during DDM training using hierarchical k-means on DINOv2 embeddings of the training set. Cluster distance computation. For each routing decision at timestep t , we compute the Euclidean distance from the current sample’s DINOv2 embedding to each of the 8 cluster centroids. The cluster rank is determined by sorting these distances (rank 1 = closest). For Top- k routing, we report the minimum rank among selected experts. Velocity alignment computation. For each expert k at each timestep, we compute the velocity prediction v_{k}(x_{t}) and measure its cosine similarity with the blended velocity v_{\text{blend}}(x_{t})=\sum_{j\in\mathcal{S}}w_{j}v_{j}(x_{t}) . This requires evaluating all 8 experts at each recorded timestep, increasing computational cost by approximately 4\times compared to standard Top-2 sampling. Statistical testing. Differences between selected and non-selected expert alignment scores are tested using paired t -tests, pairing by sample and timestep. Correlations are reported as Spearman’s \rho with two-tailed p -values. Appendix D Sensitivity Analysis Details This appendix provides additional details for the trajectory sensitivity analysis in Section 6 . Table 6 reveals that the router term \sum_{k}v_{t}^{(k)}\nabla_{x}w_{t}^{(k)} dominates the expert term by 2–4 orders of magnitude across all routing strategies. This dominance reflects the inherent sensitivity of softmax routing to input perturbations, a property shared by Top-2 and full ensembling alike. However, when comparing routing strategies, the router term provides limited discriminative signal: it is uniformly large regardless of how many experts are selected. The expert term \sum_{k}w_{t}^{(k)}\nabla_{x}v_{t}^{(k)} , by contrast, captures how the weighted mixture of expert outputs changes with input. For this reason, our trajectory-local sensitivity traces (Figure 2 ) report the expert-only Jacobian \|\sum_{k}w_{t}^{(k)}\nabla_{x}v_{t}^{(k)}\| , isolating the component that explains sensitivity differences between routing strategies. The router gradient dominance is documented separately in Figure 4 and Table 6 . D.1 Experimental Setup We use the pretrained DDM Paris model ( 11 ) via released pretrained checkpoints. The model consists of K=8 experts trained on a subset of LAION-Aesthetics ( 20 ) . The dataset was partitioned into 8 semantic clusters via two-stage hierarchical k-means on DINOv2-ViT-L/14 embeddings. The model uses a DiT-B/2 router ( \sim 129M parameters) and 8 DiT-XL/2 experts ( 19 ) (a modified version of the DiT-XL/2 experts with \sim 606M parameters each, \sim 5B total). We compare Top-1, Top-2, and full-ensemble routing strategies on the DDM architecture ( 17 ) . To avoid circularity, that is defining failure via L_{\text{eff}} and then claiming L_{\text{eff}} predicts failure, we use Eq.( 2 ) as a methodologically independent error signal. We set \tau_{\text{refine}} as the 99th percentile of \Delta_{\text{refine}} on Top-2 runs, then apply this fixed threshold across all methods. This avoids tuning thresholds to match \widehat{L}_{\text{eff}}^{(h)} predictions. Appendix F presents additional experiments that test the robustness of our main findings. D.2 Experiment 1: Local Truncation Error Following Section 6.2.1 , we estimate one-step local truncation error by comparing a single Heun step of size h ( 13 ) against a higher-precision reference obtained by subdividing the same interval into 10 Heun sub-steps of size h/10 . For each routing strategy, we sample 1,000 trajectories and evaluate at randomly selected trajectory points (x_{t},t) the maximum local truncation error defined as \epsilon^{\text{local}}=\max_{n}\|\epsilon_{n}^{\text{local}}(t_{n})\|=Ch^{p+1} for some constant C . Table 5 reports these measurements. Table 5 : Local truncation error is routing-invariant. One-step Heun local error \epsilon_{n}^{\text{local}} at h{=}0.01 and h{=}0.005 , with empirical scaling. Mean \pm std. over n{=}1000 trajectories per routing strategy. Routing \boldsymbol{\epsilon^{\textbf{local}}} ( h{=}0.01 ) \boldsymbol{\epsilon^{\textbf{local}}} ( h{=}0.005 ) Scaling Top-1 0.539 \pm 0.045 0.304 \pm 0.034 1.776 \times Top-2 0.542 \pm 0.046 0.306 \pm 0.034 1.773 \times Full (8) 0.543 \pm 0.048 0.307 \pm 0.035 1.771 \times Local error is essentially identical across routing strategies, confirming that routing does not affect single-step numerical accuracy. D.3 Experiment 2: Trajectory-Local Sensitivity We track \|J_{x}v(\tilde{x}^{(h)}_{t_{n}},t_{n})\| across time for Top-1, Top-2, and full ensemble, and separate norms for selected vs. suppressed experts. For a routed vector field v_{t}(x_{t})=\sum_{k}w_{t}^{(k)}(x_{t})v_{t}^{(k)}(x_{t}) , the Jacobian satisfies \nabla_{x}v_{t}=\sum_{k}w_{t}^{(k)}\nabla_{x}v_{t}^{(k)}+\sum_{k}v_{t}^{(k)}\nabla_{x}w_{t}^{(k)}. This decomposition separates sensitivity from experts ( \sum_{k}w_{t}^{(k)}\,\nabla_{x}v_{t}^{(k)} ) from sensitivity from routing ( \sum_{k}v_{t}^{(k)}\nabla_{x}w_{t}^{(k)} ). Since \sum_{k}w_{k}=1 implies \sum_{k}\nabla_{x}w_{k}=0 , the router term can be rewritten as \sum_{k}(v_{k}-v_{t})\nabla_{x}w_{k} , showing that its magnitude is governed by inter-expert disagreement times router sensitivity—large \|\nabla_{x}w_{k}\| alone does not inflate this term if experts agree. We report norms of each term separately as a diagnostic; by the triangle inequality, the full Jacobian norm satisfies \|\nabla_{x}v_{t}\|\leq\|J_{\text{expert}}\|+\|J_{\text{router}}\| , but these bounds need not be tight. Table 6 reports the two terms evaluated at t{=}0.5 (mean \pm std. over n{=}100 trajectories). Table 6 reveals the router term dominates by 2–4 orders of magnitude, but is similar across strategies. Since this shared dominance cannot explain the quality differences observed between routing strategies, the expert term \sum_{k}w_{k}\nabla_{x}v_{k} —which captures how the weighted mixture of expert outputs responds to input perturbations—is the relevant quantity for understanding routing-quality relationships. We therefore report the expert-only Jacobian in Figure 2 . Table 6 : Jacobian decomposition at t{=}0.5 ( n{=}100 samples). Both terms are measured along sampling trajectories. The router gradient term dominates for both strategies. Full ensemble shows nominally higher router term means, though the difference is small relative to variance. Strategy \|J_{\text{expert}}\| \|J_{\text{router}}\| Dominant Top-2 routing 7.58 \pm 1.53 923 \pm 1.4K Router Full ensemble (8) 7.58 \pm 1.59 1161 \pm 2.1K Router Figure 4 : Temporal profile of the two Jacobian terms under Top-2 and full ensembling. Left: Expert term \|\sum_{k}w_{k}\nabla_{x}v_{k}\| (linear scale). Right: Router term \|\sum_{k}v_{k}\nabla_{x}w_{k}\| (log scale). The router term dominates by 2–4 orders of magnitude, but both routing strategies show similar router contributions. D.4 Experiment 3: Step-Size Refinement We compute \Delta_{\text{refine}} by running N{=}50 and 2N{=}100 steps from the same initial noise x_{1} and prompt, decoding both final latents with D(\cdot) , and measuring LPIPS in image space. Figure 3 shows the correlation between \widehat{L}_{\text{eff}}^{(h)} and \Delta_{\text{refine}} . Appendix E Limitations Correlation strength. The weak correlations between L_{\text{eff}} and \Delta_{\text{refine}} in our main in-distribution setting ( \rho<0.1 ) limit the predictive utility of trajectory Jacobian analysis for identifying failure cases. Future work should explore whether alternative sensitivity metrics (e.g., integrated sensitivity, switching frequency) provide stronger predictive signals. Unexplained typical-set attractivity. Our convergence argument relies on a trajectory-local boundedness condition ( L_{\text{eff}}(x_{1})<\infty ), and our empirical results are consistent with trajectories remaining in moderate-sensitivity regions. However, we do not provide a proof that the exact probability flow dynamics must enter and remain in such low-sensitivity regions, nor do we characterize basins of attraction for the routed field. Developing a dynamical explanation is an important direction for future work. Scope. This paper makes a mechanistic claim about decentralized expert systems: routing trades off numerical sensitivity and expert-data alignment, and alignment can dominate quality. Because this claim is established by controlled comparisons that hold the expert pool and router fixed, it does not require external diffusion baselines. Adding external models without matched training would primarily answer a different question, and could obscure the routing mechanism due to unavoidable training/data confounds. Appendix F Extra Experiments This subsection presents additional experiments that test the robustness of our main findings. F.1 Full-ensemble tuning variants. We test whether the full ensemble’s sensitivity metrics can be improved via inference-only modifications that keep the expert pool fixed and require no retraining: (i) temperature scaling of router logits sweeping T\in\{0.1,0.25,0.5,1.0,2.0,4.0\} , and (ii) top- p truncation of the router distribution, keeping the smallest set of experts whose cumulative mass exceeds p and renormalizing within this set. Table 7 summarizes the best variants from each sweep. Neither modification substantially changes \widehat{L}_{\text{eff}}^{(h)} or \Delta_{\text{refine}} compared to the baseline full ensemble. Table 7 : Full-ensemble rescue attempts (no retraining). Temperature scaling uses w_{T}=\mathrm{softmax}(z/T) ; we report the best T from a sweep over \{0.1,0.25,0.5,1.0,2.0,4.0\} . Top- p truncation keeps the smallest set of experts whose cumulative probability exceeds p and renormalizes. Results from n{=}50 samples per configuration. Variant \boldsymbol{\widehat{L}_{\textbf{eff}}^{(h)}} \downarrow \boldsymbol{\Delta_{\textbf{refine}}} \downarrow Full ensemble (baseline) 15.97 \pm 5.51 0.043 \pm 0.052 Temp scaling ( T{=}4.0 ) 15.00 \pm 4.71 0.044 \pm 0.056 Top- p truncation ( p^{\star}{=}0.9 ) 15.85 \pm 5.30 0.043 \pm 0.052 F.2 Counterfactual Routing We evaluate two inference-only counterfactuals for the full ensemble (Table 8 ): (i) weight clipping that suppresses experts whose Jacobian norms are above the median at the current state, and (ii) Misaligned Top-2 (random expert selection), which preserves sparsity while explicitly breaking proximity-based alignment. Table 8 : Counterfactual routing interventions. Mean \Delta_{\text{refine}} from Eq. 2 (computed without Jacobian metrics). Results from n{=}50 samples. Condition \boldsymbol{\Delta_{\textbf{refine}}} \downarrow Top-2 (base) 0.043 \pm 0.054 Full ensemble (8) 0.039 \pm 0.052 Full + weight clip ( \|\nabla_{x}v_{k}\|<\text{median} ) 0.037 \pm 0.029 Misaligned Top-2 (random) 0.040 \pm 0.035 Table 9 : Temperature scaling sweep for full ensemble routing. Lower temperatures sharpen the weight distribution toward top experts; higher temperatures flatten it. All metrics measured on n{=}100 samples. T{=}4.0 achieves the lowest \Delta_{\text{refine}} in this sweep. T Entropy \boldsymbol{\widehat{L}_{\textbf{eff}}^{(h)}} \boldsymbol{\Delta_{\textbf{refine}}} 0.10 0.17 18.84 \pm 7.43 0.060 \pm 0.047 0.25 0.43 18.68 \pm 7.07 0.054 \pm 0.038 0.50 0.79 18.06 \pm 5.65 0.054 \pm 0.051 1.00 1.24 17.32 \pm 5.48 0.050 \pm 0.049 2.00 1.70 16.35 \pm 4.56 0.047 \pm 0.045 4.00 1.96 16.17 \pm 4.88 0.044 \pm 0.034 F.3 Failure Mode Analysis To understand how different routing strategies fail, we categorize samples by three failure indicators: high routing uncertainty, poor numerical convergence, and high effective Lipschitz constant. Table 13 reports the frequency of each failure mode across routing strategies. Table 10 : Top- p truncation sweep for full ensemble routing ( T{=}1.0 ). Lower p values exclude low-probability experts. All metrics measured on n{=}50 samples. Truncation has minimal effect on \widehat{L}_{\text{eff}}^{(h)} or \Delta_{\text{refine}} . p \boldsymbol{\widehat{L}_{\textbf{eff}}^{(h)}} \boldsymbol{\Delta_{\textbf{refine}}} 0.8 16.51 \pm 5.64 0.043 \pm 0.052 0.9 15.85 \pm 5.30 0.043 \pm 0.052 1.0 15.97 \pm 5.51 0.043 \pm 0.052 F.4 Full-Ensemble Rescue Attempts We test inference-time modifications to the full ensemble to investigate whether its poor sample quality (despite superior numerical stability) can be improved without retraining. As discussed in Section 6 , full ensemble has the lowest \widehat{L}_{\text{eff}}^{(h)} and \Delta_{\text{refine}} but worst FID due to expert-data misalignment: experts process out-of-distribution inputs from data partitions they were not trained on. We test inference-time temperature scaling of router logits: w_{T}(x,t)=\mathrm{softmax}(z(x,t)/T) . We sweep T\in\{0.1,0.25,0.5,1.0,2.0,4.0\} for the full ensemble strategy ( v(x,t)=\sum_{k=1}^{K}w_{T,k}(x,t)v_{k}(x,t) ), holding the expert pool and solver fixed. Table 9 reports the results of the temperature sweep. We also sweep p\in\{0.8,0.9,1.0\} . Table 10 reports the results. Truncation has minimal effect on sensitivity metrics. F.5 Generalization Tests Without Retraining To test whether the sensitivity ordering is specific to the training distribution, we evaluate the same frozen experts and router on a held-out prompt set. We generate n{=}100 samples per routing strategy and report the same sensitivity diagnostics. Table 11 shows that the \Delta_{\text{refine}} ordering (Full lowest) persists under this prompt shift, while \widehat{L}_{\text{eff}}^{(h)} values are comparable across strategies. Table 11 : Generalization test on COCO captions (out-of-distribution prompts). Frozen experts/router, new prompt distribution. Results from n{=}100 samples with Heun solver (50 steps). \Delta_{\text{refine}} ordering matches Table 4 (Full lowest), while \widehat{L}_{\text{eff}}^{(h)} values are comparable across strategies. Strategy \boldsymbol{\widehat{L}_{\textbf{eff}}^{(h)}} \downarrow \boldsymbol{\Delta_{\textbf{refine}}} \downarrow Spearman \rho(\widehat{L}_{\text{eff}}^{(h)},\Delta_{\text{refine}}) Top-1 27.17 \pm 12.08 0.114 \pm 0.109 0.11 Top-2 26.32 \pm 11.95 0.083 \pm 0.089 0.01 Full (8) 26.37 \pm 11.83 0.048 \pm 0.067 - 0.01 We also evaluate two forms of distribution shift: (i) out-of-distribution prompts and (ii) stressed numerical regimes. We evaluate the fixed trained experts and router on COCO captions as prompts. We sample n{=}100 prompts uniformly at random from the caption set, generate one sample per prompt with fixed seeds, and compute \widehat{L}_{\text{eff}}^{(h)} and \Delta_{\text{refine}} as in the main text. We additionally report the AUC of \widehat{L}_{\text{eff}}^{(h)} for predicting high \Delta_{\text{refine}} events using the same percentile thresholding protocol. We repeat the same evaluation under a harder numerical regime without retraining: (i) fewer solver steps (Heun-25 instead of Heun-50), and (ii) higher CFG ( 10 ) (7.5 instead of 4.0). The goal is not quality, but whether sensitivity rankings and predictiveness persist. Table 14 compares baseline and stressed regimes. F.6 Switching Sensitivity: Margin and Vector-Field Gap We analyze nonsmooth expert switching by combining (i) proximity to the switching surface (router margin) and (ii) the jump size in the routed vector field (vector-field gap). This addresses the limitation that for hard Top-1 the Jacobian J_{x}v is defined only inside routing regions and does not capture discontinuities at switches. Let z_{k}(x,t) be router logits, p(k\mid x,t)=\mathrm{softmax}(z(x,t))_{k} , and let k_{(1)},k_{(2)} index the top-2 logits. We report: (1) probability margin m_{p}=p_{(1)}-p_{(2)} , (2) logit margin m_{z}=z_{(1)}-z_{(2)} , (3) vector-field gap g=\|v_{k_{(1)}}-v_{k_{(2)}}\|_{2} , (4) switching score S_{\text{switch}}=g/(m_{z}+\epsilon_{\text{sw}}) with \epsilon_{\text{sw}}=10^{-3} for numerical stability. For each trajectory we summarize by S_{\text{eff}}=\max_{t}S_{\text{switch}}(x(t),t) and an integrated variant S_{\text{int}}=\int_{0}^{1}S_{\text{switch}}(x(t),t)\,dt . For each saved trajectory state (x_{t},t) we compute (m_{p},m_{z}) from the router, then evaluate the two corresponding experts v_{k_{(1)}},v_{k_{(2)}} to obtain g and S_{\text{switch}} . For Top-1, this requires one extra expert evaluation at analysis time. We subsample timesteps identically to the L_{\text{eff}} pipeline. We first reproduce margin statistics as a proxy for distance to switching surfaces. See Table 15 . Low-margin segments concentrate in failures, but margin alone does not measure the impact of switching. We evaluate how different predictors perform at identifying failures within Top-1 routing in Table 12 . Table 12 : Comparison of failure predictors for Top-1 routing. S_{\text{eff}} (switching score) combines margin and vector-field gap. The vector-field gap g alone achieves the best prediction of high \Delta_{\text{refine}} (AUC 0.63). Top-1 predictor AUC (high \Delta_{\text{refine}} ) Spearman vs. \Delta_{\text{refine}} m_{p} only 0.50 0.07 g only 0.63 0.20 S_{\text{eff}} (margin+gap) 0.55 0.08 L_{\text{eff}} only 0.58 0.08 L_{\text{eff}}+S_{\text{eff}} 0.58 0.11 The key findings are: (1) the vector-field gap g alone is the best single predictor for high \Delta_{\text{refine}} , (2) combining L_{\text{eff}} with switching features does not improve over g alone. Table 13 : Failure mode analysis across routing strategies ( n{=}100 samples). Thresholds: Routing uncert. = max entropy {>}1.5 nats; Poor conv. = \Delta_{\text{refine}}{>}0.1 ; High L_{\text{eff}} = \widehat{L}_{\text{eff}}^{(h)}{>}50 . Poor convergence decreases with more experts. High L_{\text{eff}} events are rare. Note: Routing uncertainty naturally increases with ensemble size since more experts contribute non-zero weights. Strategy \boldsymbol{\widehat{L}_{\textbf{eff}}^{(h)}} \boldsymbol{\Delta_{\textbf{refine}}} Routing uncert. Poor conv. High L_{\text{eff}} Top-1 24.21 \pm 7.44 0.112 \pm 0.114 60% 39% 1% Top-2 23.68 \pm 8.07 0.094 \pm 0.074 64% 33% 1% Top-4 23.56 \pm 6.74 0.049 \pm 0.059 88% 11% 0% Full (8) 23.72 \pm 6.97 0.038 \pm 0.053 94% 5% 1% Table 14 : Generalization under baseline vs. stressed numerical regimes (COCO captions, n{=}100 ). Stressed regime uses fewer solver steps (Heun-25) and higher CFG (7.5). Sensitivity rankings persist across regimes. Strategy Regime \boldsymbol{\widehat{L}_{\textbf{eff}}^{(h)}} \boldsymbol{\Delta_{\textbf{refine}}} Spearman AUC Top-1 Baseline 20.09 \pm 12.61 0.131 \pm 0.128 0.09 0.56 Top-2 Baseline 18.39 \pm 5.57 0.078 \pm 0.079 0.31 0.69 Full (8) Baseline 17.29 \pm 5.82 0.054 \pm 0.073 0.05 0.52 Top-1 Stressed 18.16 \pm 5.99 0.206 \pm 0.134 0.20 0.54 Top-2 Stressed 17.27 \pm 5.53 0.190 \pm 0.148 0.03 0.59 Full (8) Stressed 16.07 \pm 4.68 0.136 \pm 0.135 - 0.06 0.48 Table 15 : Router probability margin statistics for stable vs. unstable samples (unstable = \Delta_{\text{refine}} above 75th percentile). Unstable samples exhibit lower margins and more frequent near-switching events, but margin alone is insufficient for prediction. Routing Median m_{p} % Steps m_{p}<0.05 Top-1 (stable samples) 0.42 \pm 0.18 8.3% Top-1 (unstable samples) 0.21 \pm 0.14 24.7% Top-2 (stable samples) 0.38 \pm 0.16 9.1% Top-2 (unstable samples) 0.19 \pm 0.12 27.2%