ATRI: Mitigating Multilingual Audio Text Retrieval Inconsistencies by Reducing Data Distribution Errors
Yuguo Yin, Yuxin Xie, Wenyuan Yang, Dongchao Yang, Jinghan Ru, Xianwei Zhuang, Liming Liang, Yuexian Zou · 2025
Read ATRI: Mitigating Multilingual Audio Text Retrieval Inconsistencies by Reducing Data Distribution Errors by Yuguo Yin, Yuxin Xie, Wenyuan Yang, Dongchao Yang, Jinghan Ru, Xianwei Zhuang, Liming Liang, Yuexian Zou (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-14627.txt.
Description
Yuguo Yin and colleagues fix a flaw in multilingual audio-text retrieval. The same audio should match its caption just as well in any language the caption is in. Past schemes show uneven instance similarity matching across languages. The team studies the flaw in theory as two errors. One is a multilingual modal alignment direction error, and one is a weight error. They derive an upper bound for the weight error.
By study of that bound, they trace the flaw to data distribution error. It is caused by random sampling of languages in training. Their scheme, ATRI, counters it with one-to-k contrastive learning. It also uses audio-English co-anchor contrastive learning. Tests on translated forms of the AudioCaps and Clotho data sets cover eight mainstream languages, including English. The scheme hit top scores on recall and consistency.
Catalog record
| CREATOR | Yuguo Yin, Yuxin Xie, Wenyuan Yang, Dongchao Yang, Jinghan Ru, Xianwei Zhuang, Liming Liang, Yuexian Zou |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 48 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2502-14627