How to Train Your Latent Diffusion Language Model Jointly With the Latent Space
Viacheslav Meshchaninov, Alexander Shabalin, Egor Chimbulatov, Nikita Gushchin, Ilya Koziev, Alexander Korotin, Dmitry Vetrov · 2026
Read How to Train Your Latent Diffusion Language Model Jointly With the Latent Space by Viacheslav Meshchaninov, Alexander Shabalin, Egor Chimbulatov, Nikita Gushchin, Ilya Koziev, Alexander Korotin, Dmitry Vetrov (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-07933.txt.
Description
Viacheslav Meshchaninov and six co-authors train the latent space and the diffusion model together for text generation. Latent diffusion draws interest for language because it denoises whole sequences in parallel, but everything depends on building the right latent space, which is usually fixed in advance by a separate autoencoder.
Their LDLM reshapes a pretrained language model's internal forms with a trainable encoder, and trains encoder, diffusion model, and decoder jointly. Naive joint training fails, producing a poor model. Their recipe adds a mean-squared decoder loss, diffusion-to-encoder warmup, adaptive timestep sampling, and noise at the decoder input, with ablations showing each piece matters. On OpenWebText and LM1B it beat existing discrete and continuous language models while writing several times faster.
Catalog record
| CREATOR | Viacheslav Meshchaninov, Alexander Shabalin, Egor Chimbulatov, Nikita Gushchin, Ilya Koziev, Alexander Korotin, Dmitry Vetrov |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 96 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.07933