LeVo 2: Stable and Melodious Song Generation via Hierarchical Representation Modeling and Progressive Post-Training
Shun Lei, Huaicheng Zhang, Dapeng Wu, Yaoxun Xu, Lishi Zuo, Wei Tan, Hangting Chen, Guangzheng Li, Jianwei Yu, Zhiyong Wu, Dong Yu · 2026
Read LeVo 2: Stable and Melodious Song Generation via Hierarchical Representation Modeling and Progressive Post-Training by Shun Lei, Huaicheng Zhang, Dapeng Wu, Yaoxun Xu, Lishi Zuo, Wei Tan, Hangting Chen, Guangzheng Li, Jianwei Yu, Zhiyong Wu, Dong Yu (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-30642.txt.
Introduction
Shun Lei and ten colleagues present LeVo 2, a system that generates full-length songs, presented in 2026 as a hybrid of a language model and a diffusion model. Song generation must keep a track coherent and musical, render vocals and accompaniment in fine detail, and follow lyrics and prompts. Mixed-token systems plan well but blur track details. Dual-track systems sound better but plan weakly over long sequences.
LeVo 2 splits the job by level. Its language model first predicts mixed tokens for overall planning, then vocal and accompaniment tokens in parallel for detail, while a diffusion-based codec rebuilds the waveform. Training adds an automated aesthetics scorer that tiers training songs by musicality before preference alignment begins. Staged post-training then improves quality, controllability and musicality in separate passes. In expert listening tests, LeVo 2 beat open-source baselines on six subjective dimensions and neared leading commercial systems.
Catalog record
| CREATOR | Shun Lei, Huaicheng Zhang, Dapeng Wu, Yaoxun Xu, Lishi Zuo, Wei Tan, Hangting Chen, Guangzheng Li, Jianwei Yu, Zhiyong Wu, Dong Yu |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 56 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2606.30642
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
LeVo 2: Stable and Melodious Song Generation via Hierarchical Representation Modeling and Progressive Post-Training Shun Lei, Huaicheng Zhang, Dapeng Wu, Yaoxun Xu, Lishi Zuo, Wei Tan, Hangting Chen, Guangzheng Li, Jianwei Yu, Zhiyong Wu, Dong Yu Abstract Full-length song generation must preserve coherence and musicality, render detailed vocal and accompaniment acoustics, and follow lyrics and prompts. Existing language model-based systems face a structural trade-off: mixed-token modeling preserves vocal-instrument coordination but obscures track-specific details, whereas dual-track prediction improves acoustics but requires longer sequences and weakens global planning. We present LeVo 2, a hybrid LLM-Diffusion framework for controllable full-length song generation. LeVo 2 formulates this trade-off as hierarchical modeling: LeLM first predicts mixed tokens for semantic planning, then predicts vocal and accompaniment tokens in parallel for track-specific refinement, while a diffusion-based Music Codec reconstructs full-length waveforms. A central contribution of this extended version is an aesthetics-guided training schedule for alignment. During pre-training, an automated music aesthetic evaluation framework assigns musicality-tier conditions to large-scale data, providing musicality priors before preference alignment. Progressive post-training applies SFT, large-scale offline DPO, and closed-loop semi-online DPO to separately improve generation quality, controllability, and musicality. Modular extension then trains the Track-Specific LM for acoustic refinement while preserving the aligned semantic planner. This schedule separates musicality learning, controllability alignment, and acoustic refinement, mitigating optimization conflict and the limitations of static offline preference pairs. Expert listening tests and objective evaluations show that LeVo 2 outperforms open-source baselines across six subjective dimensions, and approaches leading commercial systems on several listening metrics. Ablations validate the effects of the training strategy, aesthetics guidance, scaling, and hierarchical architecture. I Introduction Music serves as a fundamental pillar of human culture, with songs standing out by uniquely blending expressive vocals and rich instrumental accompaniment to convey the emotions and thoughts of creators. Driven by rapid advancements in Artificial Intelligence Generated Content (AIGC), automated music generation has become an active research topic in both academia and industry. While remarkable progress has been made in specific sub-domains—including symbolic music generation [ 48 , 10 , 34 , 50 ] , instrumental music generation [ 2 , 36 , 8 , 19 , 29 ] , and singing voice synthesis [ 5 , 15 , 53 , 27 , 13 ] —automatically generating a complete song remains a substantial challenge. It requires generative models to simultaneously synthesize and seamlessly integrate expressive vocals and high-fidelity instrumental tracks while preserving musicality and following instructions. This task is further complicated when generating full-length songs, where maintaining lyric alignment and long-term structural coherence become more demanding. Early attempts in song generation, such as Jukebox [ 9 ] , utilized language models (LMs) to predict discrete codes—referred to as “mixed tokens”—extracted from the combined audio of vocals and accompaniment. Subsequent studies [ 23 , 20 , 46 , 16 , 47 ] further optimized LM architectures and modeling paradigms to enhance overall musicality. Nevertheless, mixed token-based approaches exhibit critical limitations. Specifically, the restricted discrete vocabulary and the mutual acoustic masking of audio tracks obscure fine-grained details and hinder high-fidelity generation. To address this, YuE [ 51 ] introduces a dual-track token prediction strategy to separately model vocal and accompaniment tracks. SongGen [ 28 ] extends this idea and shows that the interleaved token prediction strategy effectively reduces interference between different token types. While these methods produce high-quality individual tracks, their independent prediction mechanisms often struggle to maintain vocal-instrument harmony. Furthermore, the interleaved paradigm drastically increases the sequence length, severely limiting model scalability and exacerbating the difficulty of maintaining musicality and following instructions. Concurrently, diffusion-based systems, exemplified by the DiffRhythm [ 32 , 4 , 17 ] and ACE-Step [ 12 , 11 ] series, avoid discrete tokens but encounter challenges in accurately capturing temporal phoneme-acoustic alignments and maintaining long-term structural consistency across full-length songs. Recent industry song generation tools [ 37 , 41 , 31 , 30 ] have also emerged, but none of these systems has disclosed its methodology or training strategy. Beyond architectural limitations, the song generation community has long been constrained by the scarcity of high-quality data and noisy annotations. On the one hand, while models trained on such datasets can successfully fit the general music distribution, they lack the musicality priors required to align the generated songs with human preferences. On the other hand, unreliable data annotations impair the model’s instruction following capabilities, leading to lyrical hallucinations and unstable controllability. To mitigate these issues, recent studies have turned to preference alignment. DiffRhythm+ [ 4 ] employs Direct Preference Optimization (DPO) to enhance musicality, while LeVo [ 22 ] , DiffRhythm 2 [ 17 ] , and HeartMuLa [ 47 ] further explore multi-preference alignment strategies to simultaneously improve both musicality and instruction-following abilities. Nevertheless, these existing approaches exhibit two limitations. First, simultaneously optimizing multiple preference objectives often induces gradient conflicts. The mutual interference between different objectives may lead to an averaging effect, which restricts the upper bound of each individual capability. Second, these methods rely exclusively on static, offline paired datasets. This not only restricts the model’s performance upper bound within the capabilities of the pre-trained models used for dataset construction, but also renders the alignment process susceptible to reward hacking. LeVo 2 treats the trade-off between global song coherence and track-level acoustic detail as a hierarchical modeling problem. Its core component, LeLM, separates global semantic planning from track-specific acoustic refinement through two coordinated token prediction tasks. The Mixed Semantic LM predicts mixed tokens to capture song-level musical structure, including melody, rhythm, tempo, and vocal-instrument coordination. Conditioned on the hidden states of this global semantic planner, the Track-Specific LM predicts vocal and accompaniment tokens in parallel, recovering fine-grained acoustic details while avoiding the sequence-length expansion caused by interleaved dual-track prediction. The diffusion-based Music Codec then reconstructs the predicted tokens into full-length song waveforms. The main training contribution of LeVo 2 is a decoupled, aesthetics-guided alignment strategy for full-length song generation. Training a controllable song generator requires different learning signals for musicality, lyric alignment, prompt consistency, and track-level acoustic refinement. LeVo 2 assigns these objectives to separate stages instead of optimizing them under a single preference objective. During pre-training, LeVo 2 scales model capacity and training data, includes pure instrumental music to strengthen accompaniment modeling, and uses an automated music aesthetic evaluation framework to assign musicality-tier conditions. These conditions provide musicality priors before preference alignment. During progressive post-training, SFT first narrows the generation distribution toward high-quality songs, large-scale offline DPO then improves lyric alignment and prompt consistency, and closed-loop semi-online DPO further improves musicality using samples from the evolving policy. This schedule mitigates optimization conflict and the limitations of static offline preference pairs. Finally, modular extension trains the Track-Specific LM to refine vocal and accompaniment acoustics while preserving the aligned global semantic planner. Extensive experiments demonstrate the superiority of our approach. Through a large-scale expert subjective evaluation, we comprehensively assessed LeVo 2 across six core dimensions: Overall Musicality, Melody, Arrangement, Instrumental Sound Quality, Vocal Sound Quality, and Structure. LeVo 2 significantly outperforms all open-source baselines across all dimensions and approaches leading closed-source systems on several listening metrics. Objective evaluations further validate its capabilities in lyric alignment and versatile control over genres, emotions, and instruments. Audio examples, inference code, and full model