BNMusic: Blending Environmental Noises into Personalized Music

Chi Zuo, Martin B. Møller, Pablo Martínez-Nuevo, Huayang Huang, Yu Wu, Ye Zhu · 2025

Read BNMusic: Blending Environmental Noises into Personalized Music by Chi Zuo, Martin B. Møller, Pablo Martínez-Nuevo, Huayang Huang, Yu Wu, Ye Zhu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-10754.txt.

Introduction

Chi Zuo and five co-authors present a system that turns annoying noise into music, in a 2025 paper. The usual fix for noise is masking, burying it under a louder, nicer sound, but a poor match in rhythm forces the volume up. Their system, BNMusic, instead grows music out of the noise itself, shaped by a text prompt from the user.

The method works in two stages. First it composes a full piece of music that carries the character of the recorded noise, using a picture-like form of sound. Then it raises the fitting parts of the music just enough to make the noise fade from notice, while keeping the music pleasant. Tests on three sound collections showed the blends hid noise well and stayed enjoyable, which the authors offer as a kinder way to live with noisy surroundings.

Catalog record

CREATORChi Zuo, Martin B. Møller, Pablo Martínez-Nuevo, Huayang Huang, Yu Wu, Ye Zhu
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE46 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2506-10754

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

BNMusic: Blending Environmental Noises into Personalized Music Chi Zuo, Martin B. Møller, Pablo Martínez-Nuevo, Huayang Huang, Yu Wu, Ye Zhu Abstract While being disturbed by environmental noises, the acoustic masking technique is a conventional way to reduce the annoyance in audio engineering that seeks to cover up the noises with other dominant yet less intrusive sounds. However, misalignment between the dominant sound and the noise—such as mismatched downbeats—often requires an excessive volume increase to achieve effective masking. Motivated by recent advances in cross-modal generation, in this work, we introduce an alternative method to acoustic masking, aiming to reduce the noticeability of environmental noises by blending them into personalized music generated based on user-provided text prompts. Following the paradigm of music generation using mel-spectrogram representations, we propose a B lending N oises into Personalized M usic (BNMusic) framework with two key stages. The first stage synthesizes a complete piece of music in a mel-spectrogram representation that encapsulates the musical essence of the noise. In the second stage, we adaptively amplify the generated music segment to further reduce noise perception and enhance the blending effectiveness, while preserving auditory quality. Our experiments with comprehensive evaluations on MusicBench, EPIC-SOUNDS, and ESC-50 demonstrate the effectiveness of our framework, highlighting the ability to blend environmental noise with rhythmically aligned, adaptively amplified, and enjoyable music segments, minimizing the noticeability of the noise, thereby improving overall acoustic experiences. 1 Introduction Figure 1: (a): Noise cancelling is designed for individual use, requiring proximity to the user, while our noise blending aims to reduce noise annoyance for everyone in the room by seamlessly blending a complementary sound with the surrounding noise. (b): This figure demonstrates the principle of auditory masking in psychoacoustics [ 26 ] . The yellow dashed line indicates the threshold in quiet, i.e., the baseline level below which sounds are inaudible in the absence of other stimuli. When a music signal is introduced, it elevates the masking threshold in a frequency-dependent manner, shown by the yellow solid line. As a result, any concurrent noise components that fall below this elevated threshold become imperceptible to the listener, effectively masked by the presence of the music. In public environments like subway trains, passengers are often exposed to persistent and irritating noise. While active noise cancellation (ANC) [ 11 ] is effective in personal audio devices, its individual-oriented nature limits its practicality in group settings. Equipping every passenger with ANC headphones is unrealistic, and such systems often struggle with high-frequency noise. To address this, we propose a new task: rather than eliminating noise for individuals through destructive interference techniques, we aim to blend the environmental noise with correctly designed music in a way that reduces its perceptual salience for a group of listeners, as shown in Fig. 1 (a). This perceptual blending shifts the goal from directly suppressing noise to reducing its impact through harmonious audio masking, enabling a scalable auditory enhancement in shared environments, providing a more comfortable auditory experience without requiring personal devices. Beyond public transportation, this blending-based approach can also be valuable in a variety of other noise-prone environments. For instance, elevators often produce repetitive mechanical sounds during operation. Similarly, household appliances such as washing machines or automatic garage doors generate rhythmic, ongoing noise. In such settings, masking the noise with well-aligned music would help improve user comfort and overall auditory experience. Our proposed task draws inspiration from the theory of auditory masking [ 6 , 33 , 26 ] , a psychoacoustic phenomenon where the perception of one sound is reduced or eliminated by the presence of another sound. This effect is typically modeled in the frequency domain using psychoacoustic principles, which define a masking threshold—the minimum intensity below which a sound becomes inaudible to the human ear in the presence of a masker. When a new sound is introduced, it elevates the masking threshold not only at its own frequency but also across neighboring frequency bands, making weaker signals in those regions imperceptible. As shown in Fig. 1 (b), such thresholds are fundamental to many perceptual audio models, and here we use them to guide the blending of generated music with environmental noise. This task aims to diminish the perception of unwanted noise by introducing supplementary background sounds. In practice, fully masking complex environmental noise within a comfortable loudness range is rarely achievable. Instead, we aim for a more feasible and perceptually effective solution—blending. By generating music that is rhythmically aligned with the underlying noise, our approach enables partial masking while incorporating the residual noise components into the musical texture, thus reducing perceived annoyance without overwhelming the listener. To effectively blend generated music with environmental noise, it is essential that the music can perceptually mask the noise, thereby reducing its distracting impact. Achieving such blending requires that the masking effect remain strong even at relatively low overall loudness levels. To this end, the generated music should align closely with the noise in terms of rhythm or other structural properties, enabling it to integrate naturally without relying on excessive loudness. However, while recent advances in music generation—particularly those based on mel-spectrogram representations in the frequency domain [ 5 , 34 , 18 , 13 , 38 , 39 , 42 , 16 ] -have demonstrated remarkable progress, most existing models are trained to generate music from clean, structured inputs such as text prompts or orderly music excerpts. These models struggle when conditioned on noisy and unstructured inputs , as they lack the ability to retain relevant acoustic attributes from chaotic signals. Yet they also provide a natural foundation for leveraging masking principles, which are inherently frequency-dependent. Building on this emerging paradigm, we explore generating music in the frequency domain whose structure, rhythmic patterns, and statistical properties naturally align with those of the noise, enabling seamless auditory blending, which in turn diminishes the listeners’ awareness of the underlying noise. In this paper, we propose a novel approach B lending N oises into Personalized M usic (BNMusic) that uses adaptive loudness-amplified music to blend with background noise. This approach makes the noise less noticeable while maintaining a balanced overall volume, thereby enhancing the acoustic environment effectively. Our method achieves seamless alignment and integration of generated music with ambient noise through a two-stage process. In the first stage, we generate music that blends rhythmically with the noise by applying a two-step outpainting and inpainting process on the noise mel-spectrogram representation to separately reconstruct a complete piece of music in the frequency domain. Outpainting is used to generate music around the peak energy region of the noise spectrogram, followed by inpainting to reconstruct music within this high-energy zone. This sequence diffuses musical qualities from the noise’s core to the surrounding areas and back into the core, effectively integrating the noise’s inherent characteristics into a cohesive musical composition. Through this process, the method finds an optimal balance between maintaining the noise’s relevance and ensuring the resulting music remains the most pleasing to the ear. In the second stage, we dynamically amplify each music segment’s loudness to enhance the masking effect in the core area without increasing the overall volume excessively. This reduces the perceptibility of noise where its energy is most concentrated while facilitating a more organic integration of the residual

Read the complete text