Language Models Are Implicitly Continuous
Samuele Marro, Davide Evangelista, X. Angelo Huang, Emanuele La Malfa, Michele Lombardi, Michael Wooldridge · 2025
Read Language Models Are Implicitly Continuous by Samuele Marro, Davide Evangelista, X. Angelo Huang, Emanuele La Malfa, Michele Lombardi, Michael Wooldridge (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-03933.txt.
Description
Samuele Marro and co-authors argue that language models treat text as smooth flows in a 2025 paper. Language is normally stored as discrete word pieces. Yet neural networks, the engines of these models, work with smooth, unbroken math. The authors show that Transformer models learn to view a sentence as a function over a smooth space and time span.
The trait shows in most top models, among them Llama, Phi, Gemma, and Mistral lines. The team widens the Transformer math to state this time and space smoothness in input and output. Their claim hints that models grasp language in a way quite unlike people. That shift in view carries lessons for both language study and model design, they say.
Catalog record
| CREATOR | Samuele Marro, Davide Evangelista, X. Angelo Huang, Emanuele La Malfa, Michele Lombardi, Michael Wooldridge |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 66 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2504-03933