Do Factual Recall Mechanisms Carry over from Text to Speech in Multimodal Language Models?
Luca Modica Filip Landin Mehrdad Farahani Livia Qian, Gabriel Skantze Richard Johansson · 2026
Read Do Factual Recall Mechanisms Carry over from Text to Speech in Multimodal Language Models? by Luca Modica Filip Landin Mehrdad Farahani Livia Qian, Gabriel Skantze Richard Johansson (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-22170.txt.
Description
Luca Modica and co-authors ask whether factual recall mechanisms carry over from text to speech in multimodal language models. They use Causal Mediation Analysis to probe how speech language models encode, store, and retrieve factual associations. Initial results with SpiritLM, which joins discrete speech tokens with text, show gaps between text-to-text and speech-to-text recall. The mechanisms for factual recall thus carry over only partly from text to speech. The findings advance understanding of how internal knowledge mechanisms differ across modalities.
Catalog record
| CREATOR | Luca Modica Filip Landin Mehrdad Farahani Livia Qian, Gabriel Skantze Richard Johansson |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 24250 |
| LICENSE | http://creativecommons.org/licenses/by-sa/4.0/ |
| RIGHTS | CC BY SA |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.22170