From Markov to Laplace: How Mamba In-Context Learns Markov Chains
Marco Bondaschi, Nived Rajaraman, Xiuying Wei, Kannan Ramchandran, Razvan Pascanu, Caglar Gulcehre, Michael Gastpar, Ashok Vardhan Makkuva · 2025
Read From Markov to Laplace: How Mamba In-Context Learns Markov Chains by Marco Bondaschi, Nived Rajaraman, Xiuying Wei, Kannan Ramchandran, Razvan Pascanu, Caglar Gulcehre, Michael Gastpar, Ashok Vardhan Makkuva (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-10178.txt.
Description
Marco Bondaschi and seven co-authors study how the Mamba model learns from examples placed in its input. Their paper appeared in two thousand twenty-five. Mamba is a fast rival to the transformer models behind most chat tools. The team tested it on Markov chains, simple step-by-step random processes. Even a single-layer Mamba learned a classic smoothing rule for guessing the next step.
The team shows that this smoothing rule is the best possible guess in two formal senses. They explain the result through the way Mamba mixes nearby steps, a step called convolution. Tests matched the theory. The authors say this is the first formal link between Mamba and a best-known statistical method. They close by pointing to new research paths the result opens.
Catalog record
| CREATOR | Marco Bondaschi, Nived Rajaraman, Xiuying Wei, Kannan Ramchandran, Razvan Pascanu, Caglar Gulcehre, Michael Gastpar, Ashok Vardhan Makkuva |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 82 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2502-10178