Cross-Modal Robustness Transfer (CMRT): Training Robust Speech Translation Models Using Adversarial Text

Abderrahmane Issam, Yusuf Can Semerci, Jan Scholtes, Gerasimos Spanakis · 2026

Read Cross-Modal Robustness Transfer (CMRT): Training Robust Speech Translation Models Using Adversarial Text by Abderrahmane Issam, Yusuf Can Semerci, Jan Scholtes, Gerasimos Spanakis (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-11933.txt.

Description

Speech translation systems are mostly tested on clean, curated data, yet real speech bends words through inflection, dialect, and non-native accents. Abderrahmane Issam and three co-authors show that leading end-to-end speech translation models are highly vulnerable to a text-based adversarial attack on inflectional morphology, adapted here to speech. Building adversarial speech data to train against such attacks is costly and technically hard.

Their answer is Cross-Modal Robustness Transfer, a framework that moves adversarial robustness from text to speech, removing the need for adversarial speech data in training. Across four language pairs, the method improves adversarial robustness by more than three BLEU points on average. It sets a new baseline for robust speech translation without the overhead of generating adversarial speech.

Catalog record

CREATORAbderrahmane Issam, Yusuf Can Semerci, Jan Scholtes, Gerasimos Spanakis
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE37 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602-11933

Browse the full library · About the Tower of Babel