Zero-Shot Mono-to-Binaural Speech Synthesis
Alon, Julian, Soroosh, RJ, Nadav, Bastiaan, Eliya · 2024
Read Zero-Shot Mono-to-Binaural Speech Synthesis by Alon, Julian, Soroosh, RJ, Nadav, Bastiaan, Eliya (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-08356.txt.
Description
A single-channel recording lacks the sense of space that two ears give, yet training a model to add that space once needed paired two-channel data. Levkovitch and co-authors present a method in 2024 that needs no such training at all. A plain geometry step gives a first draft of the two-channel sound.
That step warps time and scales loudness by where the source sits. A pre-trained voice tool then polishes the draft in rounds. The team also built a fresh test set to judge methods on room types they had not seen. On the standard set, the new method matches trained rivals by ear. On the new set, it beats them. The result hints that large pre-trained tools carry more latent skill than their training aimed at.
Catalog record
| CREATOR | Alon, Julian, Soroosh, RJ, Nadav, Bastiaan, Eliya |
|---|---|
| YEAR | 2024 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 25 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2412-08356