T2A-Feedback: Improving Basic Capabilities of Text-to-Audio Generation via Fine-grained AI Feedback

Zehan Wang, Ke Lei, Chen Zhu, Jiawei Huang, Sashuai Zhou, Luping Liu, Xize Cheng, Shengpeng Ji, Zhenhui Ye, Tao Jin, Zhou Zhao · 2025

Read T2A-Feedback: Improving Basic Capabilities of Text-to-Audio Generation via Fine-grained AI Feedback by Zehan Wang, Ke Lei, Chen Zhu, Jiawei Huang, Sashuai Zhou, Luping Liu, Xize Cheng, Shengpeng Ji, Zhenhui Ye, Tao Jin, Zhou Zhao (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-10561.txt.

Introduction

Zehan Wang and ten co-authors improve text-to-audio generation by scoring its basic skills with automated feedback. Current models handle simple sounds but often miss events in complex prompts or produce poor acoustic quality. Human preference tuning is hard to gather at scale for audio.

Their scoring pipelines check three things: whether every named event actually occurs, whether events arrive in the described order, and overall sound quality. These scores matched human judgments better than existing metrics. With them, the team built a preference dataset of tens of thousands of prompts and hundreds of thousands of clips. Training on that feedback targets prompt-following and fidelity together.

Catalog record

CREATORZehan Wang, Ke Lei, Chen Zhu, Jiawei Huang, Sashuai Zhou, Luping Liu, Xize Cheng, Shengpeng Ji, Zhenhui Ye, Tao Jin, Zhou Zhao
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE38 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-10561

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

T2A-Feedback: Improving Basic Capabilities of Text-to-Audio Generation via Fine-grained AI Feedback Zehan Wang, Ke Lei, Chen Zhu, Jiawei Huang, Sashuai Zhou, Luping Liu, Xize Cheng, Shengpeng Ji, Zhenhui Ye, Tao Jin, Zhou Zhao Abstract Text-to-audio (T2A) generation has achieved remarkable progress in generating a variety of audio outputs from language prompts. However, current state-of-the-art T2A models still struggle to satisfy human preferences for prompt-following and acoustic quality when generating complex multi-event audio. To improve the performance of the model in these high-level applications, we propose to enhance the basic capabilities of the model with AI feedback learning. First, we introduce fine-grained AI audio scoring pipelines to: 1) verify whether each event in the text prompt is present in the audio ( Event Occurrence Score ), 2) detect deviations in event sequences from the language description ( Event Sequence Score ), and 3) assess the overall acoustic and harmonic quality of the generated audio ( Acoustic&Harmonic Quality ). We evaluate these three automatic scoring pipelines and find that they correlate significantly better with human preferences than other evaluation metrics. This highlights their value as both feedback signals and evaluation metrics. Utilizing our robust scoring pipelines, we construct a large audio preference dataset, T2A-FeedBack , which contains 41k prompts and 249k audios, each accompanied by detailed scores. Moreover, we introduce T2A-EpicBench , a benchmark that focuses on long captions, multi-events, and story-telling scenarios, aiming to evaluate the advanced capabilities of T2A models. Finally, we demonstrate how T2A-FeedBack can enhance current state-of-the-art audio model. With simple preference tuning, the audio generation model exhibits significant improvements in both simple (AudioCaps test set) and complex (T2A-EpicBench) scenarios. The project page is available at https://T2Afeedback.github.io 1 Introduction Recent Text-to-Audio (T2A) generation models ( Huang et al., 2023b ; Huang et al., 2023a ; Liu et al., 2023a ; Liu et al., 2024 ; Ghosal et al., 2023 ; Majumder et al., 2024 ; Vyas et al., 2023 ) have made drastic performance improvements. By trained on massive audio-text data ( Gemmeke et al., 2017 ; Fonseca et al., 2021 ; Chen et al., 2020 ; Kim et al., 2019 ) , these generative models learn to generate diverse sounds with a given language prompt. For audio generation, generating harmonious multi-event audio or describing a story with audio has important applications in music ( Agostinelli et al., 2023 ) , advertising, video-audio generation ( Luo et al., 2024 ; Wang et al., 2024 ) , etc. However, as shown in Figure. , existing audio generation models are struggling to generate harmonious and high-quality audio from narrative and complex descriptions, which limits the potential for high-level applications. The failure of the generated results is often demonstrated in three aspects: 1) cannot fully include all the events described, 2) cannot accurately follow the order of all the events described, and 3) cannot organize all the events harmoniously. Therefore, the model performance in multi-event scenarios is determined by its capabilities in these three fundamental aspects. To improve the model’s performance across more advanced applications, we focus on strengthening the audio generation model’s fundamental abilities. Inspired by feedback learning in large language models ( Ouyang et al., 2022 ; Bai et al., 2022 ; Touvron et al., 2023 ) , we propose creating an audio preference dataset centered on three abilities necessary for generating harmonic and complex audio: 1) Event Occurrence Prompt-Following , 2) Event Sequence Prompt-Following , and 3) Acoustic&Harmonic Quality . Based on the preference information, we can refine the model’s core abilities, resulting in better results in both simple and challenging scenarios. However, due to the scarcity of audio data and the challenges of annotating the scale of user preferences, it is difficult to collect massive audio preference datasets that only rely on human annotators. To fill this void, we explore using AI feedback ( Cui et al., 2023 ; Lee et al., 2023 ; Yuan et al., 2024 ; Burns et al., 2023 ) in text-to-audio generation, utilizing AI models to rank audios instead of relying on human annotators. Compared to manual annotation, automating the data collection and annotation process reduces the cost of obtaining audio preference data and enhances scalability. Specifically, we develop three AI scoring pipelines to evaluate the generated audio in a fine-grained and holistic manner, corresponding to three core capabilities: • Event Occurrence Score : To specifically check whether each event occurs in, we calculate the audio-text semantic matching score for each described event separately. A lower score suggests that the corresponding event might be absent from the audio. • Event Sequence Score : To verify the correctness of event order, we analyze the start and end times of each event and compare them with the event order outlined in the text prompt. A higher score implies a greater similarity between the event sequences in caption and audio. • Acoustics&Harmonic Quality : Drawing inspiration from aesthetic scoring methods used in image quality scoring, we manually annotate acoustic and harmonic quality for audio samples. These data are then used to train an automatic acoustic&harmonic predictor. We confirm that our three scoring functions show a stronger correlation with human evaluations compared to existing automatic audio evaluation methods ( Wu et al., 2023b ; Xie et al., 2024 ) . Consequently, in addition to their application in ranking preference data, these scoring functions can be used as evaluation metrics that more effectively capture human preferences across different aspects. Leveraging these advanced AI scoring pipelines, we establish a comprehensive data collection and annotation framework. As a result, we construct T2A-Feedback , a large audio preference dataset comprising 41,627 captions and 249,762 generated audios, each annotated with detailed scores. Furthermore, to evaluate the higher-level capabilities of text-to-audio models in multi-event scenarios, we introduce a more challenging benchmark, T2A-EpicBench , which features longer, more imaginative, and story-telling captions for audio generation. We enhance the advanced text-to-audio diffusion model, Make-an-Audio 2 ( Huang et al., 2023a ) , with T2A-Feedback. Our results show that using T2A-Feedback not only effectively improves the basic capabilities of the model in simple AudioCaps benchmark, but also emergently improves the performance in complex T2A-EpicBench. 2 Related Work 2.1 Text-to-Audio Generation Text-to-audio generation is an emerging field that aims to convert textual descriptions into corresponding audio outputs. Existing text-to-audio generation methods can be divided into two categories: Diffusion-based and Language model-based. Diffusion-based techniques have gained prominence for generating high-quality, realistic audio by modeling the process of denoising. These methods, like Make-an-Audio ( Huang et al., 2023b ; Huang et al., 2023a ) , AudioLDM ( Liu et al., 2023a ; Liu et al., 2024 ) , Tango ( Ghosal et al., 2023 ; Majumder et al., 2024 ) , start with random noise and iteratively refine it to produce coherent audio over a series of denoising steps. On the other hand, Language model-based methods ( Borsos et al., 2023 ; Agostinelli et al., 2023 ; Cideron et al., 2024 ) tokenize audios as acoustic discrete tokens, and predict the tokens within an auto-regressive model conditioned on text inputs. The above models acquire the ability to generate diverse audio by training on large-scale audio-text datasets. However, current datasets like AudioSet ( Gemmeke et al., 2017 ) ,

Read the complete text