Cross-Modal Robustness Transfer (CMRT): Training Robust Speech Translation Models Using Adversarial Text
Abderrahmane Issam, Yusuf Can Semerci, Jan Scholtes, Gerasimos Spanakis · 2026
Read Cross-Modal Robustness Transfer (CMRT): Training Robust Speech Translation Models Using Adversarial Text by Abderrahmane Issam, Yusuf Can Semerci, Jan Scholtes, Gerasimos Spanakis (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-11933.txt.
Description
Speech translation systems are mostly tested on clean, curated data, yet real speech bends words through inflection, dialect, and non-native accents. Abderrahmane Issam and three co-authors show that leading end-to-end speech translation models are highly vulnerable to a text-based adversarial attack on inflectional morphology, adapted here to speech. Building adversarial speech data to train against such attacks is costly and technically hard.
Their answer is Cross-Modal Robustness Transfer, a framework that moves adversarial robustness from text to speech, removing the need for adversarial speech data in training. Across four language pairs, the method improves adversarial robustness by more than three BLEU points on average. It sets a new baseline for robust speech translation without the overhead of generating adversarial speech.
Catalog record
| CREATOR | Abderrahmane Issam, Yusuf Can Semerci, Jan Scholtes, Gerasimos Spanakis |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 37 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2602-11933