A Foundation Model for Instruction-Conditioned In-Context Time Series Tasks

Anish Saha, Konstantin Shmakov · 2026

Read A Foundation Model for Instruction-Conditioned In-Context Time Series Tasks by Anish Saha, Konstantin Shmakov (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-22586.txt.

Introduction

Saha and Shmakov present iAmTime, a foundation model for time series tasks. The model learns from written instructions plus input and output examples shown at test time. It does not need its weights retrained for each new task. Its design pairs a multi-scope transformer encoder with a task-aware patch decoder. Training covers forecasting, filling gaps, sorting, anomaly spotting, and signal splitting. The authors report that iAmTime beats strong rival models on zero-shot forecasting tests. It also matches or beats them on four tasks that are not forecasting.

Catalog record

CREATORAnish Saha, Konstantin Shmakov
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE91985
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.22586

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

A Foundation Model for Instruction-Conditioned In-Context Time Series Tasks Anish Saha, Konstantin Shmakov Abstract In-context learning (ICL) is an established paradigm in large language models, enabling task adaptation at inference time through contextual examples rather than parameter updates. While several recent time-series foundation models incorporate forms of contextual conditioning or example-based prompting, these approaches typically rely on implicit positional context, retrieval, or task-specific objectives, rather than explicit instruction-conditioned demonstrations. We propose a foundation model for instruction-conditioned in-context time-series tasks, built on a quantile regression variant of a T5 encoder–decoder architecture, in which historical “examples” and “queries” are represented using a structured, prompt-like tokenization scheme with specialized semantic tokens that explicitly distinguish target series, covariates, context, and task-specific future information. A hierarchical Transformer architecture comprising per-example encoding, example-level fusion, and cross-example attention enables the model to condition its decoding behavior on in-context input–output demonstrations, allowing it to infer forecasting strategies and other time-series tasks without task-specific fine-tuning. The model is trained on a large-scale corpus of real and synthetic time series generated via mixup augmentation and controlled multivariate relationship synthesis, and combines supervised forecasting objective with self-supervised instruction-conditioned tasks such as imputation, reconstruction, classification, and source de-mixing. We show that this multi-task, example-conditioned training encourages the model to learn a distribution over time-series tasks and mappings, improving its ability to adapt to local structure at inference time. Across datasets spanning diverse domains, frequencies, and horizons, the proposed approach consistently outperforms strong foundation baselines on both point and probabilistic forecasting benchmarks, including fev-bench, and GIFT-Eval, while also achieving competitive performance on classification and anomaly detection tasks, demonstrating the effectiveness of structured in-context prompting and hierarchical fusion for flexible, instruction-following time-series foundation models. 1 Introduction Time series modeling underpins decision-making across a wide range of domains, including retail demand forecasting, energy and load planning, financial risk management, transportation, and industrial operations. These applications rely not only on accurate forecasting, but also on related tasks such as anomaly detection ( 4 ) , regime classification ( 17 ) , imputation ( 9 ) , and scenario analysis ( 20 ) . Over the past decades, the field has evolved from classical statistical approaches ( 7 ) to advanced deep learning techniques ( 37 ) introducing a number of capable large-scale time-series foundation models ( 43 ; 13 ; 3 ) , trained on diverse datasets, that better capture nonlinear dynamics, long-range dependencies, and complex covariate interactions to achieve strong zero-shot forecasting performance across domains and temporal granularities. In parallel, in-context learning (ICL) has become an established paradigm in large language models, enabling task adaptation at inference time through contextual input–output examples rather than parameter updates. This behavior was first systematically demonstrated in large-scale language models ( 8 ) , and has since been extended through prompting strategies such as few-shot prompting ( 8 ) , instruction tuning ( 41 ) , and chain-of-thought reasoning ( 42 ) , which substantially improve zero-shot and few-shot performance without fine-tuning. Beyond empirical success, recent work has begun to clarify the mechanisms underlying ICL: 19 show that Transformers trained on sequences of input–label pairs can implement behaviors such as linear predictors, gradient-descent-like updates, and algorithm selection purely through attention, while 28 demonstrate that explicitly training on structured demonstration–query episodes substantially improves ICL ability. In this work, we bring in-context learning using instruction-conditioned demonstrations to time-series modeling. We propose a foundation model for in-context time-series tasks using a structured, prompt-like tokenization scheme (represented by ”examples” and ”queries”) with specialized semantic tokens that explicitly distinguish target series, covariates, historical context, and task-specific future information. We show that this explicit structure is necessary to handle the (i) heterogeneous temporal dynamics, (ii) variable numbers of covariates, and (iii) probabilistic multi-horizon outputs that characterize real-world time-series problems. Recent time-series foundation models such as TimesFM/TimesFM-2.5 ( 13 ; 12 ) , ICTSP ( 27 ) , Chronos-2 ( 2 ) , TOTO ( 11 ) , and TiRex ( 5 ) have already explored various forms of contextual conditioning, including long historical windows, retrieval of related series, and example-based inputs. However, in these approaches context is used to improve a fixed task, typically forecasting, instead of functioning as explicit input–output demonstrations that define a task mapping at inference time, as in NLP ICL an in our proposed work. This distinction offers the promise of recovering many benefits of fine-tuning such as domain adaptation and task specialization purely through inference-time context. To endow our model with strong in-context learning capability, we train it on a large-scale corpus of real and synthetic time series using a combination of (1) supervised forecasting objectives, and (2) self-supervised tasks, including imputation, reconstruction, classification, anomaly detection, and source de-mixing. This training regime exposes the model to a distribution of time-series tasks and mappings, encouraging it to learn how to adapt behavior based on contextual demonstrations at inference time ( 28 ) . While our approach does not employ an explicit meta-learning formulation with inner-loop optimization ( 18 ) , it can be interpreted as an amortized form of meta-learning, in which task adaptation is learned during pretraining and executed through a single forward pass at inference time. This formulation addresses a common practical scenario: a practitioner may deploy a foundation model trained primarily for forecasting, only to later encounter the need for related tasks such as anomaly detection, regime classification, or domain-specific forecasting behavior. Rather than collecting new labels and fine-tuning the model, our approach allows these requirements to be expressed directly through in-context examples. Empirically, we show that this instruction-conditioned ICL framework yields strong improvements in both point and probabilistic forecasting accuracy across diverse benchmarks, including fev-bench ( 39 ) , and GIFT-Eval ( 1 ) , while also supporting classification and anomaly detection within the same unified architecture. Our main contributions are: • Instruction-conditioned in-context learning for time series. We introduce a principled formulation of instruction-conditioned in-context learning for time-series data, in which tasks are specified through structured input–output demonstrations at inference time rather than through task-specific heads or fine-tuning. • To enable example-conditioned inference and prevent representation leakage we propose a hierarchical-encoder decoder architecture together with a structured, prompt-like tokenization scheme that explicitly encodes semantic roles (targets, covariates, historical context, and task-specific future information). • Amortized meta-learning for time-series foundation models. We introduce a new training paradigm for time-series foundation models that combines forecasting based supervision with other self-supervised tasks to enable the model to adapt to new tasks and domains purely via in-context examples at inference time. • Self-supervised tasks for pretraining time-series foundation model. We design a suite of self-supervised tasks for time-series pretraining including imputation, reconstruction, classification, anomaly detection, and source de-mixing. We also discuss data augmentation strategies and methods to introduce controlled multivariate relationships in time-series. • We empirically demonstrate the benefits of (i) instruction-conditioned training, (ii) hierarchical fusion, and (iii) amortized meta-learning, through extensive ablations and analyses. The proposed model achieves competitive or superior zero-shot performance on popular point and probabilistic forecasting benchmarks, while also exhibiting competitive performance on classification and anomaly detection tasks using the same architecture and inference procedure. 2 Related Work Table 1 : Comparison of modeling capabilities and in-context learning (ICL) usage in recent time-series foundation models. While

Read the complete text