Transformers and Their Roles as Time Series Foundation Models

Dennis Wu, Yihan He, Yuan Cao, Jianqing Fan, Han Liu · 2025

Read Transformers and Their Roles as Time Series Foundation Models by Dennis Wu, Yihan He, Yuan Cao, Jianqing Fan, Han Liu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-03383.txt.

Description

Dennis Wu, Yihan He, Yuan Cao, Jianqing Fan, and Han Liu analyze transformers as foundation models for time series. Their paper appeared in two thousand twenty-five. Such models promise one system for many forecasting tasks. The authors ask what these models can in principle learn. Theory on approximation and generalization is their lens. Experiments back the proofs.

They first show some transformers can fit autoregressive models by gradient descent. The input case is a single series. They then study MOIRAI, a model handling many covariates at once. It too can fit autoregressive structure automatically, for any count of inputs. Generalization bounds follow when data meets a mixing condition named for Dobrushin. The results help explain the design and success of these models.

Catalog record

CREATORDennis Wu, Yihan He, Yuan Cao, Jianqing Fan, Han Liu
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE101 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2502-03383

Browse the full library · About the Tower of Babel