Cross-Space Distillation: Teaching One-Step Students with Modern Diffusion Teachers
Anh Nguyen *, Ngan Nguyen *, Duc Vu *, Trung Dao, Viet Nguyen, Quan Dao, Kien Nguyen, Chi Tran, Phong Nguyen, Khoi Nguyen, Cuong Pham, Dimitris Metaxas, Vishal M. Patel, and Anh Tran · 2026
Read Cross-Space Distillation: Teaching One-Step Students with Modern Diffusion Teachers by Anh Nguyen *, Ngan Nguyen *, Duc Vu *, Trung Dao, Viet Nguyen, Quan Dao, Kien Nguyen, Chi Tran, Phong Nguyen, Khoi Nguyen, Cuong Pham, Dimitris Metaxas, Vishal M. Patel, and Anh Tran (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-32020.txt.
Description
Nguyen and thirteen colleagues tackle a mismatch that blocks efficient image generation: strong diffusion teachers and compact student models often live in different latent spaces. One-step distillation methods assume teacher and student share latent resolution and the same variational autoencoder. Modern teachers such as SD 3.5 and Flux break that assumption for small deployable students like SD 1.5, so their knowledge cannot transfer directly.
The team formalizes this setting as Cross-Space Distillation and adds a lightweight interface called Bridge. Bridge maps student latents into teacher space using a frozen student decoder as a spatial prior plus a small learnable projector, leaving the student backbone untouched. Distilled this way, SD 1.5 improves from 5.4 to 9.4 on the HPSv3 quality score while keeping one-step inference.
Catalog record
| CREATOR | Anh Nguyen *, Ngan Nguyen *, Duc Vu *, Trung Dao, Viet Nguyen, Quan Dao, Kien Nguyen, Chi Tran, Phong Nguyen, Khoi Nguyen, Cuong Pham, Dimitris Metaxas, Vishal M. Patel, and Anh Tran |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 58 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2606.32020