Few-shot Acoustic Synthesis with Multimodal Flow Matching
Amandine Brunetto · 2026
Read Few-shot Acoustic Synthesis with Multimodal Flow Matching by Amandine Brunetto (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-19176.txt.
Introduction
Brunetto introduces FLAC, a method that builds the sound of a room from very few recordings. It learns a spread of likely room impulse responses, not just one fixed answer. A diffusion transformer shapes each response from cues on space, shape, and sound. FLAC can place sound at any point in a new room. With one recording, it beats prior eight-shot methods on two data sets. A new joint sound-and-shape score, AGREE, helps judge the results.
Catalog record
| CREATOR | Amandine Brunetto |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 75951 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.19176
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Few-shot Acoustic Synthesis with Multimodal Flow Matching Amandine Brunetto Abstract Generating audio that is acoustically consistent with a scene is essential for immersive virtual environments. Recent neural acoustic field methods enable spatially continuous sound rendering but remain scene-specific, requiring dense audio measurements and costly training for each environment. Few-shot approaches improve scalability across rooms but still rely on multiple recordings and, being deterministic, fail to capture the inherent uncertainty of scene acoustics under sparse context. We introduce fl ow-matching ac oustic generation (FLAC), a probabilistic method for few-shot acoustic synthesis that models the distribution of plausible room impulse responses (RIRs) given minimal scene context. FLAC leverages a diffusion transformer trained with a flow-matching objective to generate RIRs at arbitrary positions in novel scenes, conditioned on spatial, geometric, and acoustic cues. FLAC outperforms state-of-the-art eight-shot baselines with one-shot on both the AcousticRooms and Hearing Anything Anywhere datasets. To complement standard perceptual metrics, we further introduce AGREE, a joint a coustic– g eomet r y e mb e dding, enabling geometry-consistent evaluation of generated RIRs through retrieval and distributional metrics. This work is the first to apply generative flow matching to explicit RIR synthesis, establishing a new direction for robust and data-efficient acoustic synthesis. Project page: https://amandinebtto.github.io/FLAC/ 1 Introduction Every room shapes the way we hear: a lecture hall amplifies a speaker’s voice, while a cathedral envelops sound in lingering reverberation. Reproducing these rich auditory experiences is essential for creating virtual, immersive environments, where users expect sound to reflect the space. The acoustic properties of a room are encapsulated by Room Impulse Responses (RIRs), which describe the sound propagation between source-receiver pairs. RIRs allows for auralization, i.e . , transferring a room’s acoustic signature onto any sound. However, accurately modeling RIRs is challenging because they depend on complex interactions between geometry, materials, and source-listener positions. Recently, neural acoustic fields 54 ; 79 ; 44 ; 3 ; 2 ; 10 ; 83 have enabled spatially continuous RIRs rendering in a scene. However, they must be trained for each environment using extensive RIR recordings. More scalable solutions require models that can generate RIRs in novel rooms, with minimal data and without retraining. A handful of works have explored few-shot acoustic synthesis 57 ; 34 ; 52 . These methods generate RIRs in novel environments using only a sparse set of information ( e.g . , depth maps, RGB images, sensor poses, and 8 to 20 RIR recordings) without scene-specific retraining. With limited knowledge about a new scene’s characteristics, there is no single, deterministic possible RIR: few-shot generalization is an inherently ambiguous problem. Yet, existing few-shot methods overlook this uncertainty, producing only a unique deterministic prediction. To address this, we propose FLAC, a conditional generative model for few-shot acoustic synthesis based on flow matching 47 . This framework extends diffusion models 32 ; 76 with increased performance and versatility and has demonstrated strong performance in audio 49 ; 36 ; 42 and images 18 generation. FLAC is, to the best of our knowledge, the first application of generative flow matching to explicit RIR synthesis. Rather than learning a deterministic mapping, our model estimates a distribution of plausible RIRs given sparse scene context, explicitly capturing the uncertainty inherent in few-shot scenarios. We condition the generation on multimodal context, including scene geometry around the receiver, sensor poses, and a minimal set of RIR recordings. By formulating few-shot acoustic synthesis as a conditional generative task, we enable scene-consistent sound generation in novel environments even from only one audio measurement. To assess generation quality, we complement traditionally used perceptual metrics by introducing a set of scene consistency metrics that ensure the predicted RIR matches the scene’s geometry. To this end, we introduce AGREE ( A coustic- G eomet R y E mb E dding), a CLIP-style 65 dual-encoder network that aligns RIRs and scene geometry in a shared latent space. This alignment enables zero-shot audio and geometry retrieval. We leverage this shared space to provide a geometry-consistent evaluation framework through both retrieval-based scores and distributional metrics. We evaluate FLAC on the large-scale synthetic AcousticRooms 52 dataset. It achieves state-of-the-art RIR synthesis performance, demonstrating generalization across novel source-receiver positions within known rooms, as well as in entirely unseen environments. We also validate our model’s real-world capabilities through sim-to-real transfer on the Hearing-Anything-Anywhere 83 dataset. On both datasets, FLAC outperforms current state-of-the-art methods based on 8 audio recordings with a single one. In summary, our main contributions are as follows: • We propose FLAC the first conditional generative model for few-shot RIR synthesis based on flow matching. This approach accounts for the inherent uncertainty of acoustics given sparse scene context, leading to more robust predictions. • Our approach sets a new state-of-the-art on the AcousticRooms and Hearing-Anything-Anywhere datasets, generalizing to both novel source-listener pairs and environments. FLAC outperforms prior work with 8\times fewer RIR recordings. • We introduce AGREE, a joint acoustic-geometry embedding space, and propose new scene-consistency metrics that evaluate how well predicted RIRs align with the scene geometry. 2 Related Work Audio-visual learning. Audio-visual learning enhances both acoustic and vision-related tasks, including audio spatialization ( 23 ; 88 ; 25 ; 60 ; 82 ; 39 ) , de-reverberation ( 9 ; 13 ) , RIR prediction ( 74 ; 5 ; 75 ; 57 ; 44 ; 45 ; 52 ) , depth estimation ( 14 ; 4 ; 62 ; 90 ; 87 ) , navigation ( 86 ; 22 ; 12 ; 7 ; 21 ; 24 ) , floorplan reconstruction ( 58 ; 64 ) , and pose estimation ( 12 ) . FLAC extends this line of work by leveraging depth information for scene-consistent RIR generation. Neural acoustic fields. Neural acoustic fields render RIRs at novel poses by implicitly learning a mapping from spatial coordinates to the room’s acoustic field. Some approaches incorporate physical acoustic models 79 ; 83 , others infer local geometry 54 , exploit vision cues 44 ; 45 ; 11 ; 10 or use NeRF 59 and Gaussian splatting-based 38 representations 3 ; 2 . However, these methods remain scene-specific, requiring dense recordings and retraining for each new environment. Few-shot acoustic synthesis. Few-shot methods generalize across scenes using sparse observations. FewShotRIR ( 57 ) uses 20 RGB, depth and binaural audio inputs. MAGIC ( 34 ) adds semantics by extracting features with a segmentation-pretrained U-Net 70 . More recently, xRIR ( 52 ) reduces inputs to eight audio recording and a panoramic depth map, and introduces the AcousticRooms dataset specifically designed for cross-room synthesis. All prior methods treat few-shot RIR prediction as a deterministic mapping, overlooking the ambiguity of the task. By using generative flow matching, FLAC captures the distribution of plausible RIRs given sparse context, improving generalization to new scenes even with one-shot. Audio diffusion and flow matching. Diffusion-based models have advanced text-to-audio generation across speech, music, and general sound 48 ; 49 ; 35 ; 26 ; 56 ; 19 ; 20 . Flow matching further improves synthesis efficiency 36 ; 42 ; 28 . 46 recently achieved speech binauralization via flow matching. Building on these advances, we adapt generative flow matching to RIR synthesis conditioned on few-shot scene context. Joint embedding models across modalities. Joint embedding