SLD-L2S: Hierarchical Subspace Latent Diffusion for High-Fidelity Lip to Speech Synthesis

Yifan Liang, Andong Li, Kang Yang, Guochen Yu, Fangkun Liu, Lingling Dai, Xiaodong Li, Chengshi Zheng · 2026

Read SLD-L2S: Hierarchical Subspace Latent Diffusion for High-Fidelity Lip to Speech Synthesis by Yifan Liang, Andong Li, Kang Yang, Guochen Yu, Fangkun Liu, Lingling Dai, Xiaodong Li, Chengshi Zheng (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-11477.txt.

Description

Lip-to-speech synthesis usually passes through mel-spectrograms or discrete tokens, losing information. Yifan Liang and seven co-authors offer SLD-L2S, a layered subspace latent diffusion frame that maps lip movements straight to the continuous latent space of a pretrained neural audio codec. Multiple parallel subspaces process visual forms, starting from a decomposition module.

A diffusion convolution block enhances contacts within and between subspaces, while reparameterized flow matching generates target latents at once. Speech language model and semantic losses join training, moving beyond usual flow objectives. Tests show leading making quality on multiple benchmarks, in both objective and subjective tests, surpassing methods built on middle forms.

Catalog record

CREATORYifan Liang, Andong Li, Kang Yang, Guochen Yu, Fangkun Liu, Lingling Dai, Xiaodong Li, Chengshi Zheng
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE37 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602-11477

Browse the full library · About the Tower of Babel