LLM Pretraining Shapes a Generalizable Manifold: Insights into Cross-Modal Transfer to Time Series
Alexis Roger*, Prateek Humane*, Zhenghan Tai, Gwen Legate, Andrei Mircea, Vasilii Feofanov, Irina Rish · 2026
Read LLM Pretraining Shapes a Generalizable Manifold: Insights into Cross-Modal Transfer to Time Series by Alexis Roger*, Prateek Humane*, Zhenghan Tai, Gwen Legate, Andrei Mircea, Vasilii Feofanov, Irina Rish (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-20449.txt.
Description
Roger and six co-workers ask why language-pretrained transformers transfer to time-series forecasting. A linear probe on frozen model states decodes real trajectories without paired labels, and retrieval in that space forecasts well. Pretrained starts also smooth optimization, giving coherent gradients and a strongly anisotropic loss landscape. Fine-tuning then just aligns a low-dimensional space, reusing its directions for cycles, trend and repeats rather than learning time basics from scratch.
Catalog record
| CREATOR | Alexis Roger*, Prateek Humane*, Zhenghan Tai, Gwen Legate, Andrei Mircea, Vasilii Feofanov, Irina Rish |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 71376 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.20449