OccamVTS: Distilling Vision Models to 1% Parameters for Time Series Forecasting
Sisuo Lyu, Siru Zhong, Weilin Ruan, Qingxiang Liu, Qingsong Wen, Hui Xiong, Yuxuan Liang · 2025
Read OccamVTS: Distilling Vision Models to 1% Parameters for Time Series Forecasting by Sisuo Lyu, Siru Zhong, Weilin Ruan, Qingxiang Liu, Qingsong Wen, Hui Xiong, Yuxuan Liang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-01727.txt.
Introduction
Sisuo Lyu and co-authors shrink vision models for time series forecasting in a 2025 paper. Large vision models can read series data as images and forecast well, yet they carry far more weight than the task needs. The team found that series data match only the low-level texture features of such models, not their high-level scene sense.
Their method, OccamVTS, teaches a small network using the big model as a privileged guide. Layer-by-layer matching plus feature teaching passes on the useful patterns and filters out semantic noise. The distilled model keeps just one percent of the size yet beat prior best scores. Gains were largest when training data were scarce or absent.
Catalog record
| CREATOR | Sisuo Lyu, Siru Zhong, Weilin Ruan, Qingxiang Liu, Qingsong Wen, Hui Xiong, Yuxuan Liang |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 121 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2508-01727
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
OccamVTS: Distilling Vision Models to 1% Parameters for Time Series Forecasting Sisuo Lyu, Siru Zhong, Weilin Ruan, Qingxiang Liu, Qingsong Wen, Hui Xiong, Yuxuan Liang Abstract Time series forecasting is fundamental to diverse applications, with recent approaches leverage large vision models (LVMs) to capture temporal patterns through visual representations. We reveal that while vision models enhance forecasting performance, 99% of their parameters are unnecessary for time series tasks. Through cross-modal analysis, we find that time series align with low-level textural features but not high-level semantics, which can impair forecasting accuracy. We propose OccamVTS, a knowledge distillation framework that extracts only the essential 1% of predictive information from LVMs into lightweight networks. Using pre-trained LVMs as privileged teachers, OccamVTS employs pyramid-style feature alignment combined with correlation and feature distillation to transfer beneficial patterns while filtering out semantic noise. Counterintuitively, this aggressive parameter reduction improves accuracy by eliminating overfitting to irrelevant visual features while preserving essential temporal patterns. Extensive experiments across multiple benchmark datasets demonstrate that OccamVTS consistently achieves state-of-the-art performance with only 1% of the original parameters, particularly excelling in few-shot and zero-shot scenarios. 1 Introduction Time series forecasting (TSF) is a fundamental task in machine learning, underpinning a wide range of critical applications, including energy demand prediction, financial market analysis, meteorological modeling, and traffic flow optimization ( Idrees et al. 2019 ; Kiyasseh et al. 2021 ; Xu et al. 2021 ; Bi et al. 2023 ) . The objective of TSF is to anticipate future values based on historical observations of one or more temporally evolving variables. Despite its broad utility, TSF presents significant challenges due to inherent characteristics such as non-stationarity, long-range dependencies, stochastic noise, and the simultaneous presence of localized patterns and global trends across multiple temporal scales. Deep learning has emerged as the dominant paradigm for TSF, with state-of-the-art models consistently achieving strong performance across diverse benchmarks. These methods operate directly on numerical sequences or their frequency-domain counterparts, enabling rich representations of complex temporal dynamics. Convolutional neural networks (CNNs) and Transformers, for example, are particularly adept at capturing both short-term fluctuations and long-term dependencies ( Ismail Fawaz et al. 2019 ) ; advanced models such as Autoformer ( Wu et al. 2021 ) and FEDformer ( Zhou et al. 2022 ) explicitly decompose sequences into trend and seasonal components ( Bai et al. 2018 ; Hatami et al. 2018 ; Li et al. 2019 ; Liu et al. 2022a ; Wen et al. 2023 ) . Distinct from other modalities that rely on abstract semantics, these architectures effectively model time series as structured numerical signals, ultimately grounding predictions in precise and interpretable temporal behaviors. Figure 1: Modality visualization of images (ImageNet) and time series (ECL, Weather, Electricity, ETT) via the MAE encoder. (a)-(d): Original image samples extracted from the corresponding boxes in the t-SNE plot. In parallel, vision models have recently gained significant traction as an alternative and intriguing approach to TSF, driven by the observation that humans can often see meaningful patterns in time series plots. For instance, a seasoned bond trader may visually recognize a shift in trend or momentum in a price curve and form predictive judgments based on that perception. This intuitive alignment between visual understanding and temporal inference has inspired researchers to transform time series data into image-like representations, such as spectrograms, recurrence plots, or frequency-enhanced textures ( Ni et al. 2025 ) , and directly leverage pre-trained vision models for downstream time series analysis tasks. Representative examples include TimesNet ( Wu et al. 2023a ) , which recasts sequences into two-dimensional formats for CNN processing; VisionTS ( Chen et al. 2024 ) , fine-tuning Masked Autoencoders (MAE) on visualized time series ( He et al. 2022 ) ; and TimeVLM ( Zhong et al. 2025 ) , leveraging vision-language models for multimodal forecasting. These approaches repurpose vision models’ capability to detect edges, gradients, and frequency textures for identifying temporal structures and dynamics. However, repurposing large vision models (LVMs) for TSF introduces substantial redundancy and misalignment. These models are originally architected for image data rich in semantic content – a modality that fundamentally diverges from the purely numerical signals of time series. To investigate this misalignment and understand which visual features benefit time series forecasting, we conduct t-SNE visualization of features extracted by MAE from ImageNet ( Deng et al. 2009 ) and four time-series benchmarks (ECL, Weather, Electricity, ETT), as shown in Figure 1 . The results reveal that while some visual features exhibit overlap and similarity with time series data, others show significant distributional differences between the two modalities. Fig. 1 (a) and (b) lie at the interface of temporal and visual features, exhibiting similar texture patterns, while Fig. 1 (c) and (d) appear in distant image clusters, demonstrating richer semantic content. Further analysis uncovers a distinct binary differentiation pattern: time-series features align closely with texture-rich images such as metal meshes and stage curtains, yet diverge markedly from semantically complex scenes like waterbirds and parachuting activities. This pattern suggests that vision models capture low-level textural features relevant to time series, while their high-level semantic representations not only add unnecessary complexity but also impair forecasting by introducing overfitting to irrelevant visual features. In limited-data scenarios, this redundancy becomes particularly detrimental as models latch onto spurious visual patterns instead of essential temporal dynamics. Two fundamental modality mismatches underlie this problem. The first is the positional sensitivity paradox, which arises since natural images reward translation invariance, whereas time series signals critically depend on absolute temporal position. The second is the semantic representation mismatch, which stems from vision backbones that pursue high-level object semantics absent from purely numerical temporal data. These intrinsic differences manifest as three-dimensional redundancy when deploying vision models for time series forecasting: (1) Computational inefficiency from architectures over-optimized for image resolution; (2) Representational redundancy where most parameters focus on high-level semantic distinction rather than essential temporal patterns; (3) Objective misalignment where classification features directly conflict with regression tasks. This redundancy not only wastes computational resources but also risks negative transfer through semantic noise, interfering with trend prediction. These findings prompt a pressing central question: How can we retain the useful inductive biases of vision models while eliminating components that are redundant or even detrimental to TSF? We answer in the affirmative with OccamVTS : a novel cross-modal knowledge-distillation framework that transfers only the most salient 1% of predictive information from off-the-shelf vision models into compact forecasting networks. Drawing inspiration from Occam’s razor, OccamVTS systematically prunes unnecessary complexity while preserving essential core temporal cues. Unlike existing efforts that rely on direct fine-tuning (as in VisionTS) or architectural adaptation (as in TimeVLM), our approach employs native pre-trained LVMs as privileged teachers, guiding lightweight student models via carefully designed pyramid-style feature alignment and selective distillation. This strategy differs in three critical ways: (1) It avoids inheriting architectural constraints by directly distilling from unmodified vision backbones; (2) It introduces a hierarchical pyramid-style alignment mechanism to precisely map spatial features to temporal representations; (3) It explicitly targets redundancy, with empirical results showing that removing up to 99% of vision model parameters not only retains performance, but can actually improve accuracy by systematically mitigating overfitting to irrelevant features. Through this principled simplification, OccamVTS effectively captures frequency