LaTIM : Measuring Latent Token-to-Token Interactions in Mamba Models

Hugo Pitorro, Marcos Treviso · 2025

Read LaTIM : Measuring Latent Token-to-Token Interactions in Mamba Models by Hugo Pitorro, Marcos Treviso (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-15612.txt.

Description

Pitorro and Treviso introduce LaTIM, a tool for reading how Mamba models link words in twenty twenty-five. Mamba is a state space model that rivals transformers on long texts. Unlike attention models, it lacked tools to show which tokens affect which others. That gap limited debugging and trust.

LaTIM decomposes a Mamba model's processing token by token. It works for both the first and second Mamba generations. The authors test it on translation, copying, and retrieval based generation. The method reveals the model's token to token interaction patterns clearly. Those patterns show how the model selectively uses sequence information across layers. The tool brings Mamba interpretability closer to what attention models enjoy.

Catalog record

CREATORHugo Pitorro, Marcos Treviso
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE48 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2502-15612

Browse the full library · About the Tower of Babel