Accelerating Time Series Foundation Models with Speculative Decoding DOI: XXXXXXX.XXXXXXX ISBN: 978-1-4503-XXXX-X/2018/06 CCS: Computing methodologies Machine learning CCS: Computing methodologies Neural networks CCS: Information systems Data mining CCS: Information systems Web applications
Pranav Subbaraman, Fang Sun, Yue Yao, Huacong Tang, Xiao Luo, Yizhou Sun · 2025
Read Accelerating Time Series Foundation Models with Speculative Decoding DOI: XXXXXXX.XXXXXXX ISBN: 978-1-4503-XXXX-X/2018/06 CCS: Computing methodologies Machine learning CCS: Computing methodologies Neural networks CCS: Information systems Data mining CCS: Information systems Web applications by Pranav Subbaraman, Fang Sun, Yue Yao, Huacong Tang, Xiao Luo, Yizhou Sun (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-18191.txt.
Introduction
Pranav Subbaraman and six co-authors speed up large models that forecast time series data. Such models build a forecast patch by patch, so a long forecast needs many slow passes through a big model. That cost blocks use in web apps that must reply at once, like live price or feed tools.
The fix borrows a trick from text models. A small cheap model drafts the next few patches, and the big model checks them all in one pass. A test based on how well each patch fits decides which drafts to keep, and the first bad patch is fixed on the spot. Proofs show the fast path keeps the true output. Speed rises with no loss in forecast quality.
Catalog record
| CREATOR | Pranav Subbaraman, Fang Sun, Yue Yao, Huacong Tang, Xiao Luo, Yizhou Sun |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 68 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2511-18191
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Accelerating Time Series Foundation Models with Speculative Decoding DOI: XXXXXXX.XXXXXXX ISBN: 978-1-4503-XXXX-X/2018/06 CCS: Computing methodologies Machine learning CCS: Computing methodologies Neural networks CCS: Information systems Data mining CCS: Information systems Web applications Pranav Subbaraman, Fang Sun, Yue Yao, Huacong Tang, Xiao Luo, Yizhou Sun Abstract . Modern web applications—from real-time content recommendation and dynamic pricing to CDN optimization—increasingly rely on time-series forecasting to deliver personalized experiences to billions of users. Large-scale Transformer-based models have achieved state-of-the-art performance in time-series forecasting but suffer from high computational costs, limiting their deployment in latency-sensitive web applications. To address this challenge, we propose a general inference acceleration framework that adapts speculative decoding to autoregressive time-series models. Our approach employs a smaller “draft” model to propose future time-series patches, which are then verified in parallel by a larger “target” model, reducing the number of sequential forward passes required. We address key technical challenges in adapting this technique from discrete language tokens to continuous time-series distributions, including the design of acceptance criteria for multivariate Gaussian patches and practical variants that balance efficiency with accuracy. Through experiments on time series forecasting benchmarks relevant to web applications, we demonstrate significant inference speedups while maintaining competitive accuracy. The framework requires no architectural modifications to existing foundation models, making it immediately applicable to accelerate deployed time-series forecasting systems. Our implementation can be found at https://github.com/PranavSubbaraman/STRIDE . 1. Introduction The World Wide Web operates as a vast, dynamic ecosystem where billions of users generate continuous streams of temporal data through their interactions with web services, content platforms, and digital marketplaces. Time-series forecasting has emerged as a critical capability for modern web infrastructure, enabling intelligent services to anticipate user behavior, optimize resource allocation, and deliver personalized experiences at unprecedented scale. The ability to rapidly and accurately predict future patterns from historical data directly impacts the performance and user satisfaction of web applications, making forecasting efficiency a first-order concern for web-scale deployments. Use Case Speculative Decoding Integration Latency / Throughput Goal Real-time Content Recommendation Fast draft model + Target validation Latency \leq 50 ms, Higher TPS speedup Web Traffic & CDN Optimization Simultaneous evaluation of future time steps Low latency, high throughput parallelization Figure 1. Speculative Decoding in web-scale forecasting: speedup for real-time recommendation and parallelization for CDN optimization to achieve low latency and high throughput. However, despite the critical importance of forecasting in web applications, existing approaches face significant challenges when deployed at scale. Consider several concrete scenarios where time-series prediction latency critically affects web service quality: (1) Real-time content recommendation systems ( 4 ; 33 ) must forecast user engagement patterns to pre-fetch and rank content. For instance, a video streaming platform serving 100M+ daily active users needs to predict viewing patterns across millions of content items within 10-50ms to maintain responsive recommendations. A 2\times slowdown in forecasting inference translates to degraded user experience and reduced engagement metrics. (2) Web traffic and CDN optimization ( 21 ; 23 ) requires forecasting request patterns across global edge locations. Content delivery networks must predict traffic surges at minute-level granularity to proactively allocate bandwidth and cache resources. With thousands of edge nodes and millions of content objects, even modest latency improvements compound into significant infrastructure savings. (3) Computational advertising platforms ( 19 ; 11 ) forecast click-through rates and conversion probabilities for billions of ad impressions daily. Real-time bidding systems operate under strict latency budgets (typically <100ms total), where forecasting must complete within 10-20ms to leave time for auction logic and network round-trips. Slower predictions directly reduce revenue by missing bid opportunities. (4) E-commerce demand forecasting ( 25 ; 24 ) predicts sales patterns for inventory management and dynamic pricing. A large marketplace with 10M+ SKUs must continuously update demand forecasts as new transaction data arrives, balancing forecast accuracy with computational efficiency to enable real-time pricing decisions. Figure 1 illustrates how speculative decoding addresses these challenges through two key mechanisms: speedup for latency-critical applications like real-time recommendation, and parallelization for throughput-intensive tasks like CDN optimization. These applications share three key properties that motivate our approach of applying speculative decoding—a technique originally developed for accelerating language model inference—to time-series forecasting: (i) strong short-term regularities that enable accurate draft predictions, (ii) stringent latency budgets where parallel validation can provide significant speedups, and (iii) the need to accelerate existing foundation models without requiring retraining or architectural changes. To address these challenges, transformer-based models have become widely adopted for forecasting in web applications due to their exceptional ability to capture complex temporal dependencies and their proven success in natural language processing. Modern time-series foundation models, typically based on decoder-only Transformers operating over patch tokens, achieve state-of-the-art accuracy through large-scale pre-training on diverse datasets. However, despite their remarkable performance, these models face a fundamental efficiency challenge in production environments: autoregressive generation requires sequential forward passes proportional to the forecast horizon, creating a latency bottleneck that scales poorly with prediction length. This computational burden becomes prohibitive for web-scale deployments. A foundation model with 1B+ parameters generating a 96-step forecast (e.g., next 24 hours at 15-minute intervals) may require seconds of GPU time—orders of magnitude beyond acceptable latency budgets. This gap between model capability and deployment constraints severely limits the practical impact of advanced forecasting models in real-world web systems, forcing practitioners to choose between accuracy and latency. To bridge this gap between accuracy and efficiency, we propose adapting speculative decoding (SD) ( 14 ) to the domain of continuous time-series forecasting. Our key insight is that time-series patches, like text tokens, exhibit predictable patterns that can be exploited for parallel generation. By employing a small, fast “draft” model to propose multiple future patches based on learned temporal patterns, and a large, accurate “target” model to validate these proposals in a single parallel forward pass, we can achieve significant speedups while maintaining forecast quality. This approach is particularly well-suited for web applications where: (i) temporal patterns exhibit local regularities (e.g., diurnal web traffic cycles, periodic user engagement patterns), enabling high draft acceptance rates; (ii) the latency reduction from parallel validation outweighs the overhead of running two models; and (iii) existing pre-trained foundation models can be accelerated without modification, preserving their accuracy while meeting stringent latency requirements. For instance, in content recommendation systems, user behavior often follows predictable short-term patterns that a draft model can capture, while the target model ensures long-term accuracy for personalization. While the core idea is intuitive, adapting SD from discrete language tokens to continuous time-series patches presents unique technical challenges. First, acceptance criteria must handle continuous distributions rather than discrete probabilities, requiring careful design of likelihood ratios for multivariate Gaussian patch distributions. Second, the practical variant must balance computational efficiency with forecasting accuracy, as exact residual sampling becomes prohibitive in high-dimensional settings. Third, the framework must integrate seamlessly with existing time-series foundation models that use patch-based tokenization and autoregressive generation. We address these challenges through a combination of theoretical analysis and practical engineering, achieving significant speedups while maintaining forecast quality. Our contributions can be summarized as follows: • A general SD framework for continuous time-series patches that (i) achieves high acceptance and significant wall-clock speedups (empirically: 1.5–3 \times ) with negligible accuracy loss, and (ii) applies to existing time-series foundation models without architectural changes. • Theory covering