DashengTokenizer: One layer is enough for unified audio understanding and generation Heinrich Dinkel, Xingwei Sun, Gang Li, Jiahao Mei, Yadong Niu, Xiyang Li, Yifan Liao, Jiahao Zhou, Junbo Zhang, Jian Luan Abstract This paper introduces DashengTokenizer, a continuous audio tokenizer engineered for joint use in both understanding and generation tasks. Unlike conventional approaches, which train acoustic tokenizers and subsequently integrate frozen semantic knowledge, our method inverts this paradigm: we leverage frozen semantic features and inject acoustic information. In linear evaluation across 22 diverse tasks, our method outperforms previous audio codec and audio encoder baselines by a significant margin while maintaining competitive audio reconstruction quality. Notably, we demonstrate that this acoustic injection improves performance for tasks such as speech emotion recognition, music understanding, and acoustic scene classification. We further evaluate the tokenizer’s generative performance on text-to-audio (TTA), text-to-music (TTM), and speech enhancement (SE). Our approach surpasses standard variational autoencoder (VAE)-based methods on TTA and TTM tasks, while its effectiveness on SE underscores its capabilities as a general-purpose audio encoder. Finally, our results challenge the prevailing assumption that VAE-based architectures are a prerequisite for audio synthesis. Checkpoints are available . 1 Introduction The recent surge in Generative AI has been propelled by advances in Large Language Models (LLMs) and Diffusion Models, significantly enhancing the capabilities of audio foundation models. Both of these models are in theory capable to be used for audio understanding 11 ; 66 and audio generation 59 ; 24 , a representation gap persists in practice. Understanding tasks typically rely on unidirectional encoders that produce coarse, high-dimensional semantic embeddings. In contrast, generation employs tokenizers ( discrete or continuous autoencoders ) to ensure high-fidelty reconstruction from low-dimensional acoustic features. Current literature for joint understanding and generation thus adopt one of two architectures: (I) Employing a semantic encoder alongside an independent acoustic tokenizer. While effective, this approach is computationally redundant and increases system complexity; or (II) Training a single model to capture both semantic and acoustic information. These models frequently prioritize reconstruction, often resulting in subpar semantic representations compared to dedicated encoders In contrast to previous single tokenizer methods, which distill high-dimensional semantic knowledge into a low-dimensional acoustic model, our approach aims to do the inverse: We embed low-dimensional acoustic information into high-dimensional semantic features. We introduce DashengTokenizer, a unified continuous audio tokenizer designed for both understanding and generation across speech, music, and environmental sound domains. Our approach freezes previous semantic knowledge from a pretrained, strong semantic encoder, and injects acoustic information from a mel-spectrogram via a linear projection. The method is simple, requiring to only train a the linear projection with an additional standard acoustic decoder. A comparison with previous works can be seen in Table 1 . DashengTokenizer performs on par with previous continuous tokenizers for reconstruction tasks, while significantly outperforming codecs and encoders in general audio understanding. Furthermore, our experiments in Text-to-Audio (TTA), Text-to-Music (TTM), and Speech Enhancement (SE) demonstrate the superior versatility of our representation for high-fidelity audio generation. A summarization of DashengTokenizer’s performance is seen in Figure 1 . Table 1 : A comparison of our work in contrast to previous audio encoders and tokenizers. Approach Type Dimension Representative work Understanding Generation Audio Codec Discrete Low VQ-VAE 53 {\color[rgb]{0.9336,0.25,0.1563}\mathbf{\times}} {\color[rgb]{0.0117,0.0117,0.0117}\mathbf{\sim}} Audio Codec + Semantic Discrete Low Mimi 13 {\color[rgb]{0.0117,0.0117,0.0117}\mathbf{\sim}} {\color[rgb]{0.0117,0.0117,0.0117}\mathbf{\sim}} Audio Encoders Continuous High Whisper 42 ✓ {\color[rgb]{0.9336,0.25,0.1563}\mathbf{\times}} Acoustic Tokenizers Continuous Low AudioLDM 34 {\color[rgb]{0.9336,0.25,0.1563}\mathbf{\times}} ✓ Ours Continuous High Ours ✓ ✓ (a) Audio Understanding Results (b) Generation Performance Figure 1 : A summarization of DashengTokenizer capabilities for understanding and generation tasks. 2 Previous works Current audio tokenization research primarily utilizes (Vector Quantized) Variational Autoencoders (VQ-VAE) to compress raw waveforms or spectrograms into low-dimensional latent spaces. While standard VAEs compress audio into a low-dimensional continuous latent, VQ-VAEs apply further quantization to produce discrete tokens suitable for sequence modeling, also known as codecs. Codecs A substantial body of literature concentrates on the compression of audio signals into discrete quantized tokens. Notably, foundational frameworks such as SoundStorm 62 , Encodec 12 and DAC 30 primarily prioritized maintaining high acoustic fidelity throughout the reconstruction process. However, with the rise of Large Language Models (LLMs) for speech and audio modeling 11 ; 14 , research has increasingly transitioned towards augmenting acoustic codecs with semantic information 13 ; 10 ; 58 ; 35 ; 61 ; 31 ; 64 ; 32 . This integration aims to provide the underlying LLM with richer contextual cues, enhancing performance in downstream understanding tasks. While effective, due to the lossy quantization process, semantic codecs are outperformed in both understanding and reconstruction performance by unified tokenizers. Unified tokenizers Developing a single representation capable of both high-level understanding and high-fidelity generation remains an open challenge. While improvements have been made in recent years for general audio encoders 54 ; 60 ; 15 ; 6 , little emphasis has been paid on making these models capable of audio generation. The work most closely related to ours is Ming-UniAudio 57 . However, DashengTokenizer differs in two fundamental aspects: (I) Domain Versatility: Ming-UniAudio is restricted to the speech domain, whereas our approach generalizes across speech, music, and environmental audio. (II) Simplicity: Ming-UniAudio requires a complex three-stage training pipeline (acoustic modeling, semantic distillation, and fine-tuning), mirroring the established semantic codec training pipeline. In contrast, our framework employs a single-stage acoustic injection via a linear projection, significantly reducing training complexity while maintaining competitive performance. 3 Methodology Dashengtokenizer injects acoustic information into rich semantic embeddings. A simplified overview between our approach and previous methods can be seen in Figure 2 . Given an input signal x\in\mathbb{R}^{s} , we first obtain a semantic features z_{\text{sem}}\in\mathbb{R}^{b\times t\times d} , where b , t , and d represent batch size, temporal length, and feature dimension respectively, extracted from a frozen pre-trained model ( \mathcal{T}_{\text{frozen}} ) as: z_{\text{sem}}=\mathcal{T}_{\text{frozen}}(x). (1) Simultaneously, we extract acoustic embeddings z_{\text{ac}}\mathbb{R}^{b\times t\times d} from the same input x , extracted via a combination of a MelSpectrogram, followed by a linear projection \phi and layer normalization: z_{\text{ac}}=\text{LayerNorm}(\phi(\text{MelSpec}(x))). (2) To ensure temporal alignment, \phi is implemented as a non-overlapping patch embedding that maps consecutive spectrogram frames to the framerate of z_{\text{sem}} . The final unified features z is computed via additive fusion: z=z_{\text{sem}}+z_{\text{ac}}. (3) We train a generator (vocoder) G to reconstruct the original input x from z , G(z)\mapsto x . The training follows a Generative Adversarial Network (GAN) framework using a Multi-Frequency Discriminator (MFD) 61 and a hinge loss objective. The generator loss \mathcal{L}_{G} is defined as: \mathcal{L}_{G}=\lambda_{\text{sem}}\mathcal{L}_{\text{sem}}+\lambda_{\text{mel}}\mathcal{L}_{\text{mel}}+\mathcal{L}_{\text{fm}}+\mathcal{L}_{\text{adv}}, (4) where \mathcal{L}_{\text{fm}} and \mathcal{L}_{\text{adv}} represent feature-matching and adversarial losses, and \mathcal{L}_{\text{mel}} is the \ell_{1} reconstruction loss in the mel-frequency domain. Crucially, we introduce a semantic preservation loss \mathcal{L}_{\text{sem}} to prevent the acoustic features from overwhelming the semantic features, which would lead to a collapse of the understanding capabilities: \mathcal{L}_{\text{sem}}=\|z_{\text{sem}}-z_{\text{ac}}\|_{2}^{2}. (5) In our experiments, we set \lambda_{\text{sem}}=45 and \lambda_{\text{mel}}=45 . As illustrated in Figure 2 , DashengTokenizer offers superior efficiency through a single-stage training pipeline that eliminates the need for multi-stage distillation (see [B]) while ensuring inference consistency by retaining the semantic encoder during deployment. Other benefits of our approach is that the acoustic feature is independent of the semantic feature, allowing generation performance to be enhanced via higher sampling rates or higher-resolution spectrograms as inputs, without retraining the semantic encoder. Figure 2 : The proposed DashengTokenizer compared to prior approaches: [A] standard acoustic Acoustic modeling using VAE, and [B] semantically distilled (VQ-)VAEs. In contrast, our approach eliminates the multi-stage training required by [B] and does not rely on a semantic decoder that is discarded during inference, thereby avoiding a train-test mismatch. 4 Experiments 4.1 Datasets Our training pipeline utilizes approximately 282k hours of diverse audio data, sampled with specific domain weights: Music (21%; Million Sound Dataset 5 , MTG-Jamendo 7 ), English Speech (21%; Emilia 22 , Yodas 33 , LibriLight 27 , CommonVoice15 3 ), Chinese Speech (40%; AISHELL-1/2/3 8 ; 17 ; 46 , Emilia), Other Languages (10%; multi-lingual Emilia 22 ), and General Sound (26%; AudioSet 19 , FSD50K 18 , AudioCaps, CochlScene 25 , ACAVCaps niuyadong2026 ). By default, all datasets are resampled to 16 kHz. Evaluation benchmarks We evaluate representations of DashengTokenizer across three distinct axes. First, audio understanding is evaluated via the X-ARES 63 benchmark, utilizing a linear probing protocol across 22 tasks in the speech, music, and sound domains. Second, reconstruction fidelity is quantified using wideband Perceptual Evaluation of Speech Quality (PESQ) 45 and Short-Time Objective Intelligibility (STOI) 51 for speech on the SEED-TTS dataset 2 . Signal reconstruction quality for music and environmental audio is measured via Mel-spectrogram (Mel-16k) and Short-Time Fourier Transform (STFT-16k) distances calculated via the DAC toolkit 30 on MUSDB18-HQ 43 and AudioSet 19 , respectively. Finally, we validate generative capabilities through downstream experiments in Speech Enhancement (Valentini 52 , DNS 49 ), Text-to-Audio (AudioCaps 29 ), and Text-to-Music (MusicCaps 1 ). For SE, we employ DNSMOS P.835 44 , NISQAv2 38 , and Spksim to assess signal quality and speaker identity preservation. For both TTA and TTM, we evaluate performance using Fréchet Audio Distance ( FAD 28 ), Fréchet Distance ( FD ), Kullback–Leibler ( KL ) divergence, and CLAPScore 55 to measure text-audio alignment. 4.2 Setup The model is trained for one million steps using the AdamW 36 optimizer with a global batch size of 256. The learning rate is initialized at 5\times 10^{-4} and decayed to 10% using a cosine schedule. 4.2.1 Architecture Configuration Our framework consists of a semantic encoder, a trained acoustic encoder and an acoustic decoder, summarized in Table 2 . We further discuss each component individually. Table 2: DashengTokenizer architecture. Attribute Semantic Encoder Acoustic Encoder Decoder Parameters 630 M 0.66 M 173 M Architecture 32 Layer Transformer 2D Conv Vocos Hidden Dim 1280 1280 1280 Input 64-bin Mel 128-bin Mel Unified ( d=1280 ) Frame Rate 25 Hz 25 Hz 25 Hz (Upsample to 50 Hz) Semantic encoder We utilize the 630M parameter encoder from MiDashengLM-7B 14 , a 32-layer Transformer pretrained for general audio understanding on public datasets. Unlike alternatives with unknown training data mixture like Whisper 42 , the MiDashengLM backbone ensures full reproducibility. It operates at a 25 Hz frame rate with d=1280 dimensional embeddings. Acoustic Encoder To inject acoustic information, we apply a 2D convolution over a 128-bin mel-spectrogram, extracted every 10 ms. We use non-overlapping patches ( 128\times 4 bins) to match the 25 Hz temporal resolution of the semantic feature. This lightweight component (0.66M parameters) followed by LayerNorm 4 produces the residual features z_{\text{ac}} . Acoustic decoder The decoder upsamples the unified 25 Hz features to 50 Hz via a 1D transposed convolution. We adopt a scaled Vocos architecture 48 (173M parameters, 12 layers, 1280 hidden dimension) to accommodate the high-dimensional latent space ( d=1280 ). 5 Results We evaluate our results against existing literature, which can be broadly categorized into three distinct approaches: neural discrete codecs, audio encoders, and continuous tokenizers. 5.1 Reconstruction evaluation We evaluate speech reconstruction quality on the Mandarin (ZH) and English (EN) subsets of the Seed-TTS benchmark 2 . Here, we compare against speech codecs and continuous acoustic tokenizers (EzAudio 21 , UniFlow-Audio 56 ). All waveform targets and model outputs are resampled to 16 kHz for consistent evaluation. The results, summarized in Table 3 , demonstrate that continuous tokenizers generally exceed the performance of discrete codecs across both language benchmarks. While high-performance codecs like DAC 30 achieve strong results, they are consistently outperformed by continuous representations in perceptual quality. Notably, our proposed model achieves competitive results, while maintaining a lower framerate (25 Hz) compared to other top-performing continuous models. Table 3 : Speech reconstruction performance across different tokenizers. Codecs are highlighted in blue , while continuous tokenizers are highlighted in red . For all metrics higher is better and best are in bold and second best are underlined . Model Framerate Seed-TTS (ZH) Seed-TTS (EN) PESQ \uparrow STOI \uparrow PESQ \uparrow STOI \uparrow SNAC 47 - 1.841 0.862 1.804 0.870 Mimi 13 12.5 2.050 0.890 2.010 0.890 XCodec 2.0 61 50 2.190 0.920 2.370 0.930 XY-Tokenizer 20 12.5 2.270 0.900 2.140 0.900 DAC 30 50 3.860 0.967 3.763 0.969 EzAudio 21 50 3.857 0.987 3.668 0.989 UniFlow-Audio 56 50 4.048 0.990 3.858 0.992 MingTok-Audio 57 50 4.210 0.980 4.040 0.980 Ours 25 4.163 0.988 4.125 0.987 Beyond speech tasks, Table 4 evaluates reconstruction performance on general environmental audio and music. Notably, our model achieves strong results for AudioSet in the Mel-16k metric with a score of 0.320, outperforming baselines. While UniFlow-Audio-VAE remains highly competitive in music reconstruction, our model demonstrates robust generalization across diverse acoustic environments, consistently placing among the top-tier performers. Table 4 : General audio (Audioset) and music (MUSDB) reconstruction performance measured with Mel and STFT distances on 16 kHz audio. Codecs are highlighted in blue , while continuous tokenizers are highlighted in red . For all metrics lower is better and best are in bold and second best are underlined . Model Framerate Audioset MUSDB18 Mel-16k \downarrow STFT-16k \downarrow Mel-16k \downarrow STFT-16k \downarrow XY-Tokenizer 12.5 0.901 2.361 0.607 1.817 DAC 50 0.664 1.986 0.965 2.352 SNAC - 1.195 2.690 1.242 2.655 XCodec 2.0 50 1.253 2.925 1.273 2.855 EzAudio 50 0.349 1.434 0.258 1.168 UniFlow-Audio 50 0.321 1.385 0.232 1.129 MingTok-Audio 50 0.462 1.768 0.432 1.616 Ours 25 0.320 1.585 0.293 1.458 5.2 Understanding Speech understanding Speech understanding performance on the X-ARES benchmark is reported across eleven tasks, being emotion recognition (Crema-D, RAV), intent classification (FSC), speaker identification (VoxC), speaker counting (LibCnt), keyword spotting (SPV1), automatic speech recognition (LS100h), gender classification (LSMF), vocal sound classification (VocS), vocal imitation classification (VocI) and language identification (VoxL33). For all discrete codecs, features are extracted as continuous latents prior to the quantization stage. Note that MingTok-Audio 57 operates in two modes: A low dimensional Acoustic VAE embedding and a high-dimensional unified embedding, both of which are evaluated here. Table 5 : Understanding Performance in the Speech Domain on the X-ARES benchmark. Codecs are highlighted in blue , while encoder models are highlighed in yellow and continuous tokenizers are highlighted in red . For all metrics higher is better and best are in bold and second best are underlined . Model CREMA-D FSC LibCnt LS100h LSMF RAV SPV1 VocI VocS VoxC VoxL33 DAC 30 43.90 2.55 43.93 0.00 91.58 36.45 15.50 2.23 42.17 13.24 7.96 Encodec 12 39.16 3.48 33.78 0.00 88.44 27.71 26.70 2.16 43.06 10.92 7.89 SemantiCodec 35 60.30 46.40 66.05 0.00 98.14 55.00 88.02 23.82 84.28 60.99 26.66 FlexiCodec 32 58.00 99.00 43.00 0.00 0.00 50.00 0.00 0.00 79.00 0.00 72.00 WavTokenizer 26 33.00 2.00 38.00 0.00 0.00 27.00 0.00 0.00 27.00 0.00 9.00 OpenBeats 6 65.90 58.50 69.90 0.00 97.30 63.00 94.40 21.40 87.70 0.00 0.00 Whisper-Large-V3 42 71.32 97.78 64.40 90.00 94.93 68.47 97.75 29.30 91.48 24.76 97.38 HuBERT 23 58.06 98.73 49.72 82.45 85.94 47.71 95.67 18.82 81.71 3.39 45.06 MSM-MAE 39 68.64 0.47 71.22 0.00 97.58 66.39 92.77 0.34 87.56 71.64 66.63 WavLM 9 45.16 96.07 50.37 67.06 75.95 32.57 89.77 10.71 71.19 4.51 40.89 SPEAR-Base 60 72.90 96.90 62.40 90.00 98.60 68.80 97.10 29.10 90.20 65.50 86.10 SPEAR-XLarge 60 78.41 98.20 69.00 88.43 98.73 76.59 97.30 31.28 92.62 79.42 90.49 Dasheng-Base 15 77.20 94.38 68.80 72.40 98.47 72.50 96.70 25.05 91.00 77.98 81.29 Dasheng 0.6B 15 79.39 97.79 69.67 69.64 98.58 78.13 97.22 29.67 92.21 84.00 86.33 EzAudio 21 38.80 1.63 37.19 0.00 84.90 25.56 12.47 1.21 35.26 5.99 6.96 UniFlow-Audio 56 39.78 1.32 37.15 0.00 74.57 23.26 11.70 1.37 34.73 6.18 6.34 MingTok-Audio (Acoustic) 57 41.22 2.82 39.42 0.00 86.91 30.83 20.34 2.45 38.63 6.97 9.88 MingTok-Audio (Unified) 57 66.85 98.58 62.92 93.45 97.10 57.08 96.14 20.64 88.59 35.28 77.44 Ours 80.56 83.39 79.98 68.71 98.06 83.06 97.35 23.92 93.15 51.16 93.60 The results in Table 5 reveal a distinct representational gap where acoustic (VQ-)VAEs like EzAudio and DAC excel at reconstruction but lack the high-level features necessary for discriminative tasks. Our proposed DashengTokenizer offers highly competitive performance, securing the best score on four tasks (CREMA-D, LibCnt, RAV, VocS) and ranks second on two others (SPV1, VoxL33). The notable performance gap of DashengTokenizer observed on FSC and LS100h likely stems from our model’s retention of acoustic variance; while this is beneficial for emotion and scene recognition, it can interfere with tasks requiring pure semantic abstraction. Crucially, unlike audio encoders such as SPEAR-XLarge, our unified framework maintains high-fidelity synthesis capabilities, effectively bridging the divide between audio understanding and generation. Sound understanding We further evaluate the representational capacity of DashengTokenizer across diverse sound understanding tasks, including spoofing detection (ASV), sound-to-text retrieval (Clo), sound/environment classification (ESC, Urb8, FSD50, F18K), and domestic sound event detection (DES). The results in Table 6 mirror the trends observed in the speech domain ( Table 5 ), where neural codecs exhibit a significant performance plateau on semantic tasks. In contrast, DashengTokenizer demonstrates performance leadership across the majority of the sound understanding benchmark. Our model achieves scores of 96.40 on ESC, 59.98 on FSD50, 86.81 on F18K, and 55.40 on DES. While our framework marginally trails SPEAR-XLarge in spoofing detection (ASV), it achieves substantial gains in sound-to-text retrieval (Clo) with a score of 5.64 . Table 6 : Understanding Performance in the Sound Domain on the X-ARES benchmark. Codecs are highlighted in blue , while continuous tokenizers are highlighted in red and encoder models are highlighed in yellow . For all metrics higher is better and best are in bold and second best are underlined . Model ASV Clo DES ESC FSD50 F18K Urb8 DAC 93.46 0.73 20.20 31.20 7.80 16.44 50.26 Encodec 95.14 0.57 3.82 16.80 4.63 6.94 40.34 SemantiCodec 93.69 1.89 48.57 81.75 34.84 26.69 84.54 FlexiCodec 93.00 0.00 17.00 30.00 0.00 0.00 0.00 WavTokenizer 95.00 0.00 16.00 19.00 0.00 0.00 0.00 OpenBeats 0.00 4.00 55.20 86.80 38.00 68.90 86.30 Whisper-Large-V3 97.94 3.10 22.64 62.45 32.00 49.56 75.74 HuBERT 96.24 1.74 3.64 37.35 17.40 26.50 57.40 MSM-MAE 93.19 3.86 0.00 88.70 44.15 48.63 84.35 WavLM 95.37 0.79 16.76 31.45 10.06 19.63 53.13 SPEAR-Base 99.20 3.65 48.63 82.25 37.96 48.00 81.46 SPEAR-XLarge 99.81 3.37 53.42 88.45 44.55 64.82 83.78 Dasheng-Base 96.29 3.30 53.20 86.90 40.90 55.70 83.50 Dasheng 0.6B 99.24 4.00 55.01 87.95 44.79 61.63 84.71 EzAudio 91.44 0.40 17.01 18.05 4.86 13.38 34.97 UniFlow-Audio 93.77 0.57 16.90 19.85 4.83 12.19 35.16 MingTok-Audio (Acoustic) 92.41 0.63 15.98 23.15 5.82 13.88 44.85 MingTok-Audio (Unified) 98.70 2.00 39.80 62.25 24.83 40.81 72.87 Ours 96.45 5.64 55.40 96.40 59.98 86.81 83.80 Music understanding The results for music understanding, presented in Table 7 , continue the trends observed in the speech and general sound domains. Encoder models generally set the strongest baseline, while neural audio codecs perform poorly. Our proposed DashengTokenizer outperforms all others approaches on three of four datasets, with a particularly large lead on the challenging MAESTRO benchmark. This result exemplifies that acoustic information present in pretrained embeddings, can be helpful in the music domain. Additional experimentation in regards to the effectiveness of acoustic injection can be seen in Table 10 . Table 7 : Understanding Performance in the Music Domain on the X-ARES benchmark. Codecs are highlighted in blue , while continuous tokenizers are highlighted in red and encoder models are highlighed in yellow . For all metrics higher is better and best are in bold and second best are underlined . Model FMA GTZAN MAESTRO NSynth DAC 35.43 54.06 12.78 38.57 Encodec 28.80 35.14 23.37 32.59 SemantiCodec 57.94 66.67 8.62 68.14 FlexiCodec 0.00 0.00 0.00 0.00 WavTokenizer 0.00 0.00 0.00 0.00 OpenBeats 62.40 85.90 0.00 58.90 Whisper-Large-V3 58.90 71.77 0.00 63.50 HuBERT 42.81 51.85 0.00 48.22 MSM-MAE 62.74 84.39 0.32 73.02 WavLM 36.11 48.14 0.00 40.14 SPEAR-Base 62.06 81.08 12.65 63.55 SPEAR-XLarge 64.16 84.59 13.63 67.85 Dasheng-Base 64.00 86.90 43.25 69.29 Dasheng 0.6B 64.34 89.49 46.81 78.39 EzAudio 30.74 40.54 17.57 32.18 UniFlow-Audio 30.17 40.74 16.44 31.73 MingTok-Audio (Acoustic) 34.74 39.94 21.38 27.81 MingTok-Audio (Unified) 54.45 71.17 25.00 57.00 Ours 66.29 89.99 57.65 77.91 5.3 Speech enhancement We evaluate the utility of DashengTokenizer for speech enhancement (SE) by employing the latent-space denoising framework described in 50 . We generate noisy speech by mixing clean samples with additive noise, then convert both into embeddings using our tokenizer. A lightweight 3-layer Transformer denoiser is trained to map these noisy embeddings back to the clean manifold, optimizing for the mean squared error between the enhanced and original clean embeddings. To ensure a fair comparison, our experimental protocol, our training and evaluation setup is identical to the one in 50 . Table 8 : Speech enhancement results on the Valentini and DNS1 datasets. Baselines are taken from 50 . Higher is better and best is in bold . Dataset Model PESQ \uparrow STOI \uparrow DNSMOS \uparrow NISQAv2 \uparrow SpkSim \uparrow P808 SIG BAK OVL Valentini Noisy 1.97 0.92 3.09 3.32 3.11 2.68 3.04 0.888 LMS 1.77 0.85 3.45 3.12 4.07 2.90 4.25 0.718 Whisper 1.31 0.80 3.39 3.47 4.12 3.22 3.83 0.406 WavLM 1.27 0.81 3.49 3.49 4.13 3.26 4.12 0.408 Dasheng Denoiser 2.32 0.90 3.52 3.45 4.14 3.22 4.51 0.783 Ours 2.62 0.92 3.50 3.33 4.08 3.09 4.58 0.814 DNS1 Noisy 1.58 0.92 3.16 3.39 2.62 2.48 2.58 0.924 LMS 1.90 0.90 3.87 3.34 4.07 3.10 3.85 0.847 Whisper 1.20 0.81 3.87 3.60 4.14 3.36 4.09 0.489 WavLM 1.20 0.83 3.91 3.61 4.18 3.40 4.41 0.486 Dasheng Denoiser 2.24 0.92 4.03 3.58 4.17 3.37 4.41 0.881 Ours 2.66 0.95 4.02 3.52 4.16 3.30 4.46 0.902 Results in Table 8 resents a comparative evaluation of our proposed DashengTokenizer against several baselines, including a log-Mel spectrogram (LMS) baseline, common speech encoder models (Whisper, WavLM), and Dasheng Denoiser 50 . Our model consistently outperforms all baselines in objective speech quality and intelligibility metrics. Specifically, on both Valentini and DNS1 datasets, DashengTokenizer significantly exceeds the performance of Dasheng Denoiser in terms of PESQ and STOI. Although certain baselines show marginal leads in specific DNSMOS sub-metrics, our model yields the highest NISQAv2 scores (4.58 and 4.46) and the highest speaker similarity across both datasets, indicating superior overall perceptual quality. 5.4 Flow based audio generation Table 9 : Flow-matching generation performance for TTA and TTM, compared with the VAE from UniFlow-Audio. Best results are marked in bold . Task Model FAD \downarrow FD \downarrow KL \downarrow CLAP Score \uparrow TTA VAE 4.27 24.72 1.84 0.412 Ours 3.06 21.23 1.56 0.438 TTM VAE 4.35 23.68 1.94 0.258 Ours 3.47 22.74 2.00 0.261 We evaluate the performance of our proposed framework on text-to-music (TTM) and text-to-audio (TTA) generation tasks. Our architecture adopts the flow-based DiT 41 paradigm established by UniFlow-Audio 56 , by replacing the baseline Variational Autoencoder (VAE) with our DashengTokenizer. Because our unified latent representation resides in a significantly higher-dimensional space ( d=1280 ) compared to the standard VAE ( d=128 ), we follow RAE 65 and scale the DiT width from 1024 to 1532. To keep the model capacity comparable to the baseline, we simultaneously reduce the DiT layers from 24 to 11, resulting in approximately 750M trainable parameters for both models. For both TTM and TTA tasks, models are trained for 200k iterations using a global batch size of 192 distributed across eight NVIDIA GPUs. We utilize LP-MusicCaps 16 and WavCaps 37 as the primary training corpora for the music and audio domains and evaluate on MusicCaps 1 and AudioCaps 29 , respectively. During inference, we use 25 inference steps with a classifier-free guidance (CFG) scale of 5.0. As presented in table 9 , our proposed method demonstrates superior generation capabilities compared to the VAE baseline across both tasks. Figure 3 : Text-to-Audio and Text-to-Music training progress of our proposed framework compared with the VAE from UniFlow-Audio. Furthermore, the training progress illustrated in Figure 3 demonstrates that our proposed DashengTokenizer converges significantly faster than the baseline VAE. Notably, DashengTokenizer reaches comparable performance to the baseline while saving at least 50k update steps across the majority of metrics. This efficiency is particularly evident for the CLAP Score and KL metrics, which achieve parity up to 190k training steps earlier. 5.5 Ablation Here we provide ablation studies, in which we evaluate the two individual semantic and acoustic features as well as the proposed unified feature for understanding and reconstruction tasks. For understanding tasks, we directly evaluate each individual feature z_{\text{sem}},z_{\text{ac}},z . For reconstruction, we utilize the decoder G to synthesize the waveform directly from the respective features. The results in Table 10 indicate that the semantic features provides the primary signal for most classification tasks, while the acoustic features alone yields poor performance. However, the unified feature significantly can enhance performance in paralinguistic (CREMA-D) and music tasks (FMA, NSynth), demonstrating that injected acoustics can complement semantic features for audio classification. Table 10 : Audio understanding performance on the X-ARES benchmark for the individual acoustic, semantic and proposed unified features. Higher is better and best is in bold . Domain Task Semantic Acoustic Unified Speech CREMA-D 80.10 44.44 80.55 FSC 98.26 2.71 83.39 LibCnt 80.79 52.34 79.98 LS100h 80.32 0.00 68.70 LSMF 98.47 88.14 98.06 RAV 87.15 33.54 83.06 SPV1 97.61 19.62 97.35 VocI 30.32 2.86 23.92 VocS 93.15 47.73 93.15 VoxC 55.51 19.62 51.16 VoxL33 75.40 34.34 74.06 Sound ASV2015 98.37 89.73 96.45 Clo 6.12 0.37 5.63 DES 52.02 21.49 55.40 ESC 96.95 34.05 96.40 FSD50 61.46 8.93 59.98 F18K 89.19 27.06 86.81 Urb8 86.23 50.02 83.80 Music FMA 65.14 41.71 66.29 GTZAN 91.19 58.85 89.99 MAESTRO 38.65 58.84 57.65 NSynth 77.59 43.16 77.91 We further ablate the importance of the injected acoustic features in DashengTokenizer, where we evaluate the reconstruction performance of each feature individually. The results in Table 11 unsurprisingly show that the semantic features alone cannot be used for effective reconstruction. However, our proposed unified feature still achieves similar performance to the acoustic feature, making it suitable for both understanding and generation tasks. Overall, these results demonstrate that our unified approach successfully bridges the gap between maintaining high-fidelity acoustic information without compromising on general-purpose audio understanding. Table 11 : Reconstruction performance for semantic, acoustic and the proposed unified features. Best are in bold . Feature Audioset MUSDB18 SEED-ZH SEED-EN Mel-16k \downarrow STFT-16k \downarrow Mel-16k \downarrow STFT-16k \downarrow PESQ \uparrow Acoustic 0.268 1.497 0.267 1.410 4.178 4.131 Semantic 3.310 4.978 3.290 5.540 1.040 1.040 Unified 0.320 1.585 0.293 1.458 4.163 4.125 6 Conclusion This paper proposed DashengTokenizer, a unified audio embedding that can be used for generation and understanding tasks. Our proposed approach trains a simple linear projection to map acoustic details into high-level semantic features. The resulting approach results in a competitive performance in terms of reconstruction quality, while also outperforming audio encoders and codecs on a plethora of understanding tasks. Further, DashengTokenizer is then applied for SE tasks, where it is shown that this tokenizer achieves superior performance compared to baselines in SE. Lastly, we show that using DashengTokenizer, training efficiency for TTA and TTM tasks largely outperform competitive VAE baselines.