LLM Pretraining Shapes a Generalizable Manifold: Insights into Cross-Modal Transfer to Time Series

Alexis Roger*, Prateek Humane*, Zhenghan Tai, Gwen Legate, Andrei Mircea, Vasilii Feofanov, Irina Rish · 2026

Read LLM Pretraining Shapes a Generalizable Manifold: Insights into Cross-Modal Transfer to Time Series by Alexis Roger*, Prateek Humane*, Zhenghan Tai, Gwen Legate, Andrei Mircea, Vasilii Feofanov, Irina Rish (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-20449.txt.

Description

Roger and six co-workers ask why language-pretrained transformers transfer to time-series forecasting. A linear probe on frozen model states decodes real trajectories without paired labels, and retrieval in that space forecasts well. Pretrained starts also smooth optimization, giving coherent gradients and a strongly anisotropic loss landscape. Fine-tuning then just aligns a low-dimensional space, reusing its directions for cycles, trend and repeats rather than learning time basics from scratch.

Catalog record

CREATORAlexis Roger*, Prateek Humane*, Zhenghan Tai, Gwen Legate, Andrei Mircea, Vasilii Feofanov, Irina Rish
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE71376
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.20449

Browse the full library · About the Tower of Babel