LLM-Guided Reinforcement Learning for Audio-Visual Speech Enhancement

Chih-Ning, Jen-Cheng, Hsin-Min, Shao-Yi, Yu, Fan-Gang · 2026

Read LLM-Guided Reinforcement Learning for Audio-Visual Speech Enhancement by Chih-Ning, Jen-Cheng, Hsin-Min, Shao-Yi, Yu, Fan-Gang (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-13952.txt.

Description

Chen and co-authors use language model feedback as the reward for audio-visual speech enhancement. Standard scores like signal-to-noise ratio track perceived quality poorly and explain little. An audio language model describes the enhanced speech in words, and a sentiment model turns that text into a rating from one to five. That rating trains the enhancement model with PPO reinforcement learning. On the AVSEC-4 dataset, the method beats supervised and DNSMOS-based baselines on quality metrics and listening tests.

Catalog record

CREATORChih-Ning, Jen-Cheng, Hsin-Min, Shao-Yi, Yu, Fan-Gang
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE21892
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.13952

Browse the full library · About the Tower of Babel