ATRI: Mitigating Multilingual Audio Text Retrieval Inconsistencies by Reducing Data Distribution Errors

Yuguo Yin, Yuxin Xie, Wenyuan Yang, Dongchao Yang, Jinghan Ru, Xianwei Zhuang, Liming Liang, Yuexian Zou · 2025

Read ATRI: Mitigating Multilingual Audio Text Retrieval Inconsistencies by Reducing Data Distribution Errors by Yuguo Yin, Yuxin Xie, Wenyuan Yang, Dongchao Yang, Jinghan Ru, Xianwei Zhuang, Liming Liang, Yuexian Zou (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-14627.txt.

Description

Yuguo Yin and colleagues fix a flaw in multilingual audio-text retrieval. The same audio should match its caption just as well in any language the caption is in. Past schemes show uneven instance similarity matching across languages. The team studies the flaw in theory as two errors. One is a multilingual modal alignment direction error, and one is a weight error. They derive an upper bound for the weight error.

By study of that bound, they trace the flaw to data distribution error. It is caused by random sampling of languages in training. Their scheme, ATRI, counters it with one-to-k contrastive learning. It also uses audio-English co-anchor contrastive learning. Tests on translated forms of the AudioCaps and Clotho data sets cover eight mainstream languages, including English. The scheme hit top scores on recall and consistency.

Catalog record

CREATORYuguo Yin, Yuxin Xie, Wenyuan Yang, Dongchao Yang, Jinghan Ru, Xianwei Zhuang, Liming Liang, Yuexian Zou
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE48 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2502-14627

Browse the full library · About the Tower of Babel