From Images to Signals: Are Large Vision Models Useful for Time Series Analysis?
Ziming Zhao, ChengAo Shen, Hanghang Tong, Dongjin Song, Zhigang Deng, Qingsong Wen, Jingchao Ni · 2025
Read From Images to Signals: Are Large Vision Models Useful for Time Series Analysis? by Ziming Zhao, ChengAo Shen, Hanghang Tong, Dongjin Song, Zhigang Deng, Qingsong Wen, Jingchao Ni (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-24030.txt.
Introduction
Ziming Zhao and co-authors ask if big vision models can help read data that change over time, in a 2025 study. Time series, such as sales or heart traces, can be drawn as charts. If a model that reads images can read those charts, it could sort or forecast the series.
The team ran the first broad test of its kind: four vision models, eight ways to draw series as images, 18 data sets, and 26 rival methods. Vision models did help to sort series into classes. They flopped at forecasts, though: the best of them worked just for set model and draw pairs. They also leaned on set forecast spans and used long past spans poorly. The authors share the map of what works to guide next work.
Catalog record
| CREATOR | Ziming Zhao, ChengAo Shen, Hanghang Tong, Dongjin Song, Zhigang Deng, Qingsong Wen, Jingchao Ni |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 88 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-24030
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
From Images to Signals: Are Large Vision Models Useful for Time Series Analysis? Ziming Zhao, ChengAo Shen, Hanghang Tong, Dongjin Song, Zhigang Deng, Qingsong Wen, Jingchao Ni Abstract Transformer-based models have gained increasing attention in time series research, driving interest in Large Language Models (LLMs) and foundation models for time series analysis. As the field moves toward multi-modality, Large Vision Models (LVMs) are emerging as a promising direction. In the past, the effectiveness of Transformer and LLMs in time series has been debated. When it comes to LVMs, a similar question arises: are LVMs truely useful for time series analysis? To address it, we design and conduct the first principled study involving 4 LVMs, 8 imaging methods, 18 datasets and 26 baselines across both high-level (classification) and low-level (forecasting) tasks, with extensive ablation analysis. Our findings indicate LVMs are indeed useful for time series classification but face challenges in forecasting. Although effective, the contemporary best LVM forecasters are limited to specific types of LVMs and imaging methods, exhibit a bias toward forecasting periods, and have limited ability to utilize long look-back windows. We hope our findings could serve as a cornerstone for future research on LVM- and multimodal-based solutions to different time series tasks. 1 Introduction Time series analysis is useful across various domains, including geoscience [ 1 ] , neuroscience [ 4 ] , energy [ 24 ] , healthcare [ 38 ] , and smart city [ 37 ] . With the significant advances of sequence modeling in the language domain, recent research attention on time series has been drawn to methods ranging from Transformer [ 49 ] to Large Language Models (LLMs) [ 18 , 61 , 20 ] . As Large Vision Models (LVMs), such as ViT [ 9 ] , BEiT [ 3 ] and MAE [ 15 ] , became successful, some emergent efforts have been invested to explore the potential of LVMs in time series modeling [ 5 ] . In these works, time series are imaged , i.e. , transformed to certain image representations [ 40 ] , as illustrated by Fig. 1 (a), then fed to an LVM to learn embeddings that can be probed for downstream tasks. These works posit that LVMs, being pre-trained on vast images, are useful in time series analysis from two perspectives: (1) for high-level ( i.e. , semantic level) tasks such as classification, imaged time series can encode distinguishable temporal patterns as semantic cues that LVMs can recognize; (2) for low-level ( i.e. , numerical level) tasks such as forecasting, the intrinsic relationship between images and time series – each row/column in an image (per channel) is a sequence of continuous pixel values that resembles a univariate time series (UTS) – makes LVMs better suited to time series tasks than LLMs since LLMs consume discrete tokens. However, in-depth connections between LVMs and time series analysis remain largely underexplored. In the past several years, the effectiveness of Transformer and LLMs for time series analysis was critically questioned by [ 56 ] and [ 44 ] in tandem. When it comes to LVMs, a similar question arises – are LVMs useful for time series analysis? To underlie future research upon LVMs, including multi-modal models that integrate imaged time series [ 62 ] , a thorough feasibility study is needed to understand LVMs’ role in time series tasks. This is precisely our goal. We comprehensively study LVMs on two representative tasks, time series classification (TSC) and time series forecasting (TSF). In a nutshell, our conclusion is cautiously positive: pre-trained LVMs are useful in TSC but pose challenges when used for TSF. The current best LVM-based forecasters, although effective, are limited to specific types of LVMs and imaging methods, exhibit bias towards forecasting periods, and have limited ability to utilize long look-back window. In this work, we choose two LVMs that are supervisedly pre-trained, i.e. , ViT [ 9 ] and Swin [ 36 ] , and two LVMs that are self-supervisedly pre-trained, i.e. , MAE [ 15 ] and SimMIM [ 54 ] , along with 8 widely used methods for imaging time series as suggested in [ 40 ] . Our analysis involves 10 datasets for TSC and 8 datasets for TSF, all are widely used benchmarks [ 2 , 52 , 65 , 41 , 56 , 44 ] . In \lx@sectionsign 3 , we introduce methods for adapting LVMs to time series tasks, including input alignment and task-specific designs. Our analysis ( \lx@sectionsign 4 ) starts with thorough comparisons between LVMs and the state-of-the-art (SOTA) baselines, including 18 classification baselines and 8 forecasting baselines. The results provide an overview on the effectiveness of LVMs, shedding light on what type of LVMs ( supervised vs. self-supervised ), which imaging method ( among 8 methods ), and what output design ( linear probing vs. pre-trained decoder ) fit which task ( classification vs. forecasting ). To figure out the source of effectiveness, we compare LVMs’ zero-shot and (fully/partially) fine-tuned performance with that of the same architecture trained from scratch, which shows the pre-trained Transformer components indeed transfer useful knowledge. By testing LVMs under different shuffling of time steps, we also find that LVMs grasp sequence modeling capability. As we observe TSF is more challenging than TSC to LVMs, further TSF-specific study is conducted. From it, we reveal the best LVM forecaster is a combination of self-supervised LVMs and a specific imaging method ( i.e. , UVH in Fig. 1 (a)). Moreover, the pre-trained decoders in self-supervised LVMs play a more critical role than their encoders in forecasting. However, the current best LVM forecasters have an inductive bias that renders them basically “combine past periods” as forecasts, thus they are prone to datasets with strong periodicity. To sum up, our contributions are as follows: • To the best of our knowledge, this is the first work to comprehensively study the feasibility of LVMs in time series analysis for both high-level and low-level predictive tasks. • We compare representative LVMs using different imaging methods on datasets of various domains, summarize the current best ways to tweak LVMs for TSC and TSF tasks, assess various aspects of the adapted LVMs, including their effectiveness in terms of pre-training, imaging, decoding, fine-tuning, architecture, temporal order of data, and computational costs, for the two tasks. • We further investigate the challenge of using LVMs for forecasting by studying individual model components, potential inductive bias, and the impact of look-back windows. We hope our findings could provide an in-depth insight of LVMs’ role in time series analysis, so as to benefit future development in this emergent area and multi-modal time series research [ 17 , 31 ] . 2 Related Work Our work share similar merits as [ 56 , 44 , 66 ] , each of which sheds important lights on a single time series task, i.e. , Transformers for TSF [ 56 ] , LLMs for TSF [ 44 ] , and LLMs for time series anomaly detection (TSAD) [ 66 ] . In contrast, our work is LVM-specific, covering more tasks with in-depth analysis. This work could be considered as a substantial complement to the prior works by adding a new