Reasoning-Aware Training for Time Series Forecasting

Md Atik Ahamed, Mihir Parmar, Palash Goyal, Chun-Liang Li, Qiang Cheng, Tomas Pfister, Jinsung Yoon · 2026

Read Reasoning-Aware Training for Time Series Forecasting by Md Atik Ahamed, Mihir Parmar, Palash Goyal, Chun-Liang Li, Qiang Cheng, Tomas Pfister, Jinsung Yoon (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-08625.txt.

Introduction

Md Atik Ahamed and seven coauthors inject language model reasoning into time series forecasting without turning numbers into text tokens. Splitting numbers into tokens breaks math relations and inflates length, while pure forecast models explain nothing. Their STRIDE framework distills reasoning traces into a light language model and projects its pooled inner states as a continuous signal inside the numeric encoder.

The add on attaches to several forecasting architectures and trains with both forecast and reasoning losses. Combined with a partner system, STRIDE tops a major forecasting leaderboard, and on a reasoning benchmark it improves both accuracy and explanation quality. Forecasting gains forecasts that a reader can follow, not just numbers to trust blindly.

Catalog record

CREATORMd Atik Ahamed, Mihir Parmar, Palash Goyal, Chun-Liang Li, Qiang Cheng, Tomas Pfister, Jinsung Yoon
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE82 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.08625

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Reasoning-Aware Training for Time Series Forecasting Md Atik Ahamed, Mihir Parmar, Palash Goyal, Chun-Liang Li, Qiang Cheng, Tomas Pfister, Jinsung Yoon Abstract Time Series Foundation Models (TSFMs) excel at numerical forecasting but operate as black boxes lacking qualitative reasoning. Conversely, applying LLMs directly to temporal data introduces a modality gap: text tokenizers fragment continuous numerical values, degrading mathematical relationships and exploding sequence lengths, leading to computational overhead. To resolve this, we introduce STRIDE (Strategic Time-series Reasoning Injected via Distilled Embeddings), a novel framework natively integrating LLM reasoning into the continuous embedding space of TSFMs. Instead of discrete tokens, STRIDE distills reasoning traces into a lightweight LLM, dynamically projecting its mean-pooled hidden states as a cross-modal prior into the target numerical encoder. The architecture is jointly optimized using cross-entropy and quantile losses. Evaluations demonstrate STRIDE establishes state-of-the-art numerical forecasting on GIFT-Eval (0.674 MASE, 0.454 CRPS) compared to TSFMs and exhibits superior in-domain and out-of-domain numerical as well as reasoning performance on TFRBench. Specifically, STRIDE acts as a plug-and-play enhancement, consistently improving diverse TSFMs (e.g., Chronos-2, Timer-S1) across various LLM configurations. Thus, injecting semantic reasoning as a continuous prior equips TSFMs with human-interpretable reasoning while fundamentally improving predictive accuracy. 1 Introduction Time Series Foundation Models (TSFMs) have recently transformed numerical forecasting by applying large-scale pretraining to temporal data, establishing new baselines in numerical forecasting ( 6 ; 5 ; 20 ; 24 ; 33 ) . However, their success relies entirely on acting as black-box mathematical extrapolators. While they effectively capture historical dependencies (e.g., trends), they fundamentally lack the ability to process semantic context, articulate predictive reasoning, or dynamically adapt to external variables such as holidays or economic shifts. Since TSFMs approximate the conditional distribution P(Y|X) using only continuous numerical representations, they suffer from a blind spot in qualitative reasoning. Motivated by this limitation, recent literature has attempted to directly apply LLMs to time series forecasting ( 37 ; 44 ) . Yet, emerging studies reveal a profound “LLM mirage”–using only LLMs for forecasting often fails to meaningfully outperform TSFMs ( 36 ) . The root cause is a modality gap where forcing continuous temporal signals through discrete text tokenizers shatter numerical values into fragmented strings, degrading mathematical precision while introducing computational bottlenecks ( 1 ; 38 ) . LLMs are fundamentally designed for semantic reasoning, not for autoregressive numerical calculation. In this work, we argue that LLMs should not be forced to act as numerical forecasters; instead, their utility lies in generating statistically grounded meaningful reasoning. To this end, we reformulate the forecasting objective to model the conditional distribution P(Y|X,R) , where R acts as an explicit reasoning prior (motivated from 3 ). To formalize this new objective, we introduce STRIDE (illustrated in Figure 1 ), a novel framework that fuses semantic reasoning priors from LLMs with continuous numerical prediction by integrating it directly into TSFMs. Figure 1: Overview of the STRIDE . (Left) Diverse training corpus of \sim 275K reasoning samples across 23 datasets. (Right) The pipeline is jointly optimized via cross-entropy and quantile losses during training, enabling the framework to simultaneously output human-interpretable reasoning and numerical forecasts during evaluation. and indicate trainable and frozen modules, respectively. STRIDE relies on cross-modal representation distillation ( 16 ; 40 ) . During training, a teacher model with access to future ground-truth data and external events generates oracle reasoning, motivated by 3 . We distill this capability into a lightweight student LLM, teaching it to articulate equivalent insights using strictly historical context. Crucially, to bypass the token-limit bottlenecks and information loss of previous works ( 38 ) , we entirely avoid passing discrete text tokens to the forecasting model. Instead, we extract the continuous mean-pooled hidden states of the student LLM and project them directly into the target TSFM’s embedding space. The entire pipeline is jointly optimized end-to-end via textual cross-entropy ( 9 ) and numerical quantile ( 6 ; 20 ) losses. We conduct comprehensive evaluations across two large-scale benchmarks to validate both the numerical and reasoning performance of STRIDE . First, on the GIFT-Eval ( 4 ) , STRIDE establishes a state-of-the-art, achieving 0.674 MASE and 0.454 CRPS, outperforming the TSFMs. Second, to assess generalizability and reasoning quality, we evaluate on TFRBench ( 3 ) , where STRIDE demonstrates superior numerical performance against the strongest baselines in both in-domain (0.615 vs. 0.765 MASE) and out-of-domain (0.724 vs. 0.778 MASE) settings. Beyond numerical precision, an LLM-as-a-Judge evaluation confirms that STRIDE achieves the highest scores across four distinct qualitative metrics, consistently generating logic-grounded reasoning that outperforms zero-shot LLM forecasters. Crucially, ablation studies reveal that STRIDE is model agnostic: it functions as a plug-and-play enhancement that consistently improves performance of diverse TSFMs (e.g., Chronos-2, Timer-S1). Thus, we hope that STRIDE motivates the development of inherently interpretable foundation models for forecasting domain. In summary, our contributions are three folds: (i) a Novel Cross-Modal Framework that utilizes a generalized latent projection mechanism to natively inject LLM reasoning into TSFM embedding spaces, entirely bypassing discrete token bottlenecks; (ii) a Reasoning Distillation Pipeline enabling a lightweight student LLM to generate high-quality analytical strategies using strictly historical context; and (iii) Better Numerical and Reasoning Performance , empirically demonstrating superior performance on GIFT-Eval, and TFRBench compared to various baselines. 2 Related Works Dataset-Specific Architectures. Prior to the advent of large-scale pretraining, the state-of-the-art in time series forecasting relied on training specialized deep learning architectures. Models such as PatchTST ( 27 ) introduced patching to preserve local temporal semantics and reduce computational complexity. iTransformer ( 22 ) inverted the traditional attention mechanism to focus on variate correlations rather than temporal tokens, while TimeMachine ( 2 ) leveraged highly efficient architectures to capture long-term dependencies with linear complexity. While these models deliver strong performance on their specific target distributions, they lack zero-shot generalization capabilities. Furthermore, they operate entirely within the numerical domain, offering no textual interpretability or reasoning to explain their predictive strategies. Time-Series Foundation Models (TSFMs). The success of the Transformer architecture in NLP has spurred the development of Time-Series Foundation Models (TSFMs) designed for zero-shot generalization. Models such as TimesFM ( 13 ) , Chronos ( 6 ; 5 ) , MOIRAI ( 42 ) , Timer-S1 ( 24 ) and Lag-Llama ( 28 ) have achieved state-of-the-art performance. These approaches typically employ specialized tokenization strategies, either patching numerical values ( 27 ) or discretizing them into vocabulary tokens ( 14 ) to process time series as a language. While effective at capturing complex autoregressive dependencies and scaling laws, these models operate largely as “black boxes.” They do not natively leverage external semantic knowledge (e.g., holiday impacts or economic shifts) or qualitative reasoning capabilities. Conversely, recent attempts to replace TSFMs entirely with general-purpose LLMs have exposed a profound modality gap ( 19 ; 10 ) , where discrete text tokenization degrades mathematical precision. Our work bridges this divide: rather than replacing TSFMs, we introduce a joint optimization framework that trains both the numerical forecaster and the language model simultaneously, utilizing a latent projection mechanism to inject the structural reasoning of the LLM directly into the continuous embedding space of TSFM. Prompt-Driven and Text-to-Text Forecasting. A growing body of work explores using general-purpose LLMs (e.g., GPT-4, Gemini, Claude) for forecasting by

Read the complete text