From Markov to Laplace: How Mamba In-Context Learns Markov Chains

Marco Bondaschi, Nived Rajaraman, Xiuying Wei, Kannan Ramchandran, Razvan Pascanu, Caglar Gulcehre, Michael Gastpar, Ashok Vardhan Makkuva · 2025

Read From Markov to Laplace: How Mamba In-Context Learns Markov Chains by Marco Bondaschi, Nived Rajaraman, Xiuying Wei, Kannan Ramchandran, Razvan Pascanu, Caglar Gulcehre, Michael Gastpar, Ashok Vardhan Makkuva (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-10178.txt.

Description

Marco Bondaschi and seven co-authors study how the Mamba model learns from examples placed in its input. Their paper appeared in two thousand twenty-five. Mamba is a fast rival to the transformer models behind most chat tools. The team tested it on Markov chains, simple step-by-step random processes. Even a single-layer Mamba learned a classic smoothing rule for guessing the next step.

The team shows that this smoothing rule is the best possible guess in two formal senses. They explain the result through the way Mamba mixes nearby steps, a step called convolution. Tests matched the theory. The authors say this is the first formal link between Mamba and a best-known statistical method. They close by pointing to new research paths the result opens.

Catalog record

CREATORMarco Bondaschi, Nived Rajaraman, Xiuying Wei, Kannan Ramchandran, Razvan Pascanu, Caglar Gulcehre, Michael Gastpar, Ashok Vardhan Makkuva
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE82 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2502-10178

Browse the full library · About the Tower of Babel