Neural Spectral Band Generation for Audio Coding
Woongjib, Byeong Hyeon, Hyungseob, Inseon, Hong-Goo · 2025
Read Neural Spectral Band Generation for Audio Coding by Woongjib, Byeong Hyeon, Hyungseob, Inseon, Hong-Goo (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-06732.txt.
Introduction
Audio coding saves bits by rebuilding high tones from low ones, but the standard trick copies bands too crudely for varied sounds. Woongjib Choi, Byeong Hyeon Kim, Hyungseob Lim, Inseon Jang, and Hong-Goo Kang swap the trick for a neural generator in a 2025 paper. Their design squeezes a small side note about the high tones at the sending end. The receiver grows the high band from that note plus the decoded low band.
Training used an adversarial setup so the rebuilt sound stays plausible to the ear. Tests built on the AAC codec beat the standard high-band method on listening quality. The neural route also used much less side data. The authors pitch it for tighter streaming budgets.
Catalog record
| CREATOR | Woongjib, Byeong Hyeon, Hyungseob, Inseon, Hong-Goo |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 23 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2506-06732
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Neural Spectral Band Generation for Audio Coding Woongjib, Byeong Hyeon, Hyungseob, Inseon, Hong-Goo Abstract Spectral band replication (SBR) enables bit-efficient coding by generating high-frequency bands from the low-frequency ones. However, it only utilizes coarse spectral features upon a subband-wise signal replication, limiting adaptability to diverse acoustic signals. In this paper, we explore the efficacy of a deep neural network (DNN)-based generative approach for coding the high-frequency bands, which we call neural spectral band generation (n-SBG). Specifically, we propose a DNN-based encoder-decoder structure to extract and quantize the side information related to the high-frequency components and generate the components given both the side information and the decoded core-band signals. The whole coding pipeline is optimized with generative adversarial criteria to enable the generation of perceptually plausible sound. From experiments using AAC as the core codec, we show that the proposed method achieves a better perceptual quality than HE-AAC-v1 with much less side information. Choi Kim Lim Jang Kang Yonsei UniversitySeoul, South Korea DaejeonSouth Korea keywords audio coding, spectral band replication, generative adversarial training † † email: [email protected], [email protected], [email protected], [email protected], [email protected] † † email: {woongzip1, bhkim98, hyungseob.lim}@dsp.yonsei.ac.kr, [email protected], [email protected] † † footnotetext: This work was supported by Electronics and Telecommunications Research Institute (ETRI) grant funded by the Korean government. [25ZC1100, The research of the basic media \cdot contents technologies] 1 Introduction The primary objective of audio coding (or audio compression) is to reduce the amount of data needed to represent audio signals while preserving the quality of the decoded output [ 1 , 2 ] . Audio coding schemes can be classified as either lossless or lossy. Lossless coding preserves the original signal exactly, whereas lossy coding achieves higher compression by permitting some quality loss. In particular, perceptual audio coding [ 2 , 3 ] , a form of lossy compression, aims to maximize the data efficiency while maintaining perceived audio quality as much as possible. These methods exploit psychoacoustics [ 4 ] —studies on the relationship between acoustic stimuli and human auditory perception—to determine the optimal bit allocation over frequency bins under a bit-budget restriction. By employing psychoacoustic models (PAMs) to assess the perceptual saliency of audio components, perceptual audio codecs assign fewer bits to less perceptually significant components, thereby achieving a higher compression ratio without noticeably degrading quality. At low bit-rates, many perceptual audio codecs [ 5 , 6 ] opt to totally discard some spectral components above a certain frequency, prioritizing the encoding of lower-frequency content. While this strategy efficiently reduces bit-rate requirements, the resulting bandwidth limitation can lead to muffled sound quality [ 7 , Chapter 5] . To address these issues, many perceptual codecs such as HE-AAC [ 8 ] , MP3Pro [ 9 ] and Opus [ 10 ] incorporate spectral band replication (SBR) [ 11 ] (or a similar method), a technique designed to perceptually encode the high-frequency components using the existing core-band signals in an efficient manner. In the encoding stage, the input audio signal is analyzed by a filterbank and the key encoding parameters such as spectral envelope information and noise level estimates are extracted by a SBR encoder to be used for reconstructing the high-frequency components from the low-frequency ones. These parameters are then quantized and transmitted to the decoder. In the decoding stage, the core codec output is separated into subband signals, and a SBR decoder reconstructs high-frequency components by replicating low-frequency subband signals into the high-frequency range. The replicated subband signals are then adjusted based on the transmitted parameters and further enhanced with sinusoids or noise if needed. Figure 1 : An overview of neural Spectral Band Generation (n-SBG). The SBG encoder extracts quantized parameters from the input audio, while the SBG decoder generates high-frequency subbands based on the transmitted parameters and coded low-frequency bands. The core decoder and embedding extractor modules are shared across the encoder and decoder sides, while the core encoder and core decoder remain fixed and not trained. The SBR is a well-established parametric approach to audio bandwidth extension (BWE), a task of reconstructing missing high-frequency components from band-limited signals. Specifically in the case of SBR, low-frequency band information along with encoded side information is utilized to generate high-frequency spectral content. In parallel, numerous DNN-based BWE [ 12 , 13 , 14 , 15 ] approaches have emerged, with the aim of generating missing high-frequency spectra from low-band inputs (i.e., blind BWE). Building upon these approaches, one may expect to replace the SBR with a DNN-based BWE model in the audio coding pipeline. However, this straightforward integration may be unsuitable for general audio signals, as the correlation between the low-band and the high-band characteristics varies depending on the content [ 16 ] . Unlike these blind approaches, where the missing high-frequency content must be inferred without prior information, audio coding has access to the full-band signal at the encoder [ 7 , Chapter 5] . This fundamental difference allows for the extraction of extra information that guides high-frequency reconstruction more accurately. Inspired by SBR, we propose neural Spectral Band Generation (n-SBG) as a new framework to integrate DNN-based BWE models into traditional audio coding pipelines for more efficient high-frequency restoration. Figure 1 provides an overview of the proposed n-SBG framework. During encoding, a feature map is extracted and quantized from the original full-band audio, then transmitted as SBG parameters. The decoder utilizes these parameters together with the core codec’s band-limited output to generate the desired high-frequency bands. The key contributions of this research can be summarized as follows: • We propose a novel framework for parametric coding of high-frequency bands at low bit-rate conditions, replacing the rule-based SBR with a DNN-based encoder-decoder architecture. • Our work lays the foundation for a new paradigm in high-frequency coding by leveraging information from core codec outputs for more efficient feature extraction and accurate reconstruction. 2 Related Works 2.1 Bandwidth Extension DNN-based blind BWE approaches typically use generative models such as diffusion models [ 12 , 13 ] or generative adversarial networks (GAN) [ 14 , 15 ] to estimate high-frequency content from low-frequency inputs, based on distributions learned from full-band audio data. These methods have been found to be particularly effective for speech signals [ 15 ] , as high-frequency patterns in speech are more predictable than those in music and general audio, but their performance for general audio has not yet been fully validated. 2.2 Neural Audio Codecs Neural audio codecs (NACs) compress signals through an encoder-quantizer-decoder pipeline, often incorporating residual vector quantization (RVQ) to discretize latent embeddings from the encoder. Representative examples such as SoundStream [ 17 ] , EnCodec [ 18 ] , and DAC [ 19 ] employ convolutional autoencoders to process time-domain audio signals. To further improve perceptual quality, these methods often incorporate adversarial training with frequency-domain discriminators. Recently, a blind BWE module was integrated into a NAC to reduce bit-rate [ 20 ] , primarily focusing on speech signals. To generalize across diverse audio signals, our approach transmits additional parameters for high-frequency generation through a dedicated encoder module. 2.3 Post-Processing with Auxiliary Information While many works aim to enhance degraded codec outputs solely based on the decoded signal itself [ 21 , 22 , 23 ] , some post-processing modules incorporate features