Transformers and Their Roles as Time Series Foundation Models
Dennis Wu, Yihan He, Yuan Cao, Jianqing Fan, Han Liu · 2025
Read Transformers and Their Roles as Time Series Foundation Models by Dennis Wu, Yihan He, Yuan Cao, Jianqing Fan, Han Liu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-03383.txt.
Description
Dennis Wu, Yihan He, Yuan Cao, Jianqing Fan, and Han Liu analyze transformers as foundation models for time series. Their paper appeared in two thousand twenty-five. Such models promise one system for many forecasting tasks. The authors ask what these models can in principle learn. Theory on approximation and generalization is their lens. Experiments back the proofs.
They first show some transformers can fit autoregressive models by gradient descent. The input case is a single series. They then study MOIRAI, a model handling many covariates at once. It too can fit autoregressive structure automatically, for any count of inputs. Generalization bounds follow when data meets a mixing condition named for Dobrushin. The results help explain the design and success of these models.
Catalog record
| CREATOR | Dennis Wu, Yihan He, Yuan Cao, Jianqing Fan, Han Liu |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 101 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2502-03383