LaTIM : Measuring Latent Token-to-Token Interactions in Mamba Models
Hugo Pitorro, Marcos Treviso · 2025
Read LaTIM : Measuring Latent Token-to-Token Interactions in Mamba Models by Hugo Pitorro, Marcos Treviso (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-15612.txt.
Description
Pitorro and Treviso introduce LaTIM, a tool for reading how Mamba models link words in twenty twenty-five. Mamba is a state space model that rivals transformers on long texts. Unlike attention models, it lacked tools to show which tokens affect which others. That gap limited debugging and trust.
LaTIM decomposes a Mamba model's processing token by token. It works for both the first and second Mamba generations. The authors test it on translation, copying, and retrieval based generation. The method reveals the model's token to token interaction patterns clearly. Those patterns show how the model selectively uses sequence information across layers. The tool brings Mamba interpretability closer to what attention models enjoy.
Catalog record
| CREATOR | Hugo Pitorro, Marcos Treviso |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 48 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2502-15612