UniCast: A Unified Multimodal Prompting Framework for Time Series Forecasting

Sehyuk Park, Soyeon Caren Han, Eduard Hovy · 2025

Read UniCast: A Unified Multimodal Prompting Framework for Time Series Forecasting by Sehyuk Park, Soyeon Caren Han, Eduard Hovy (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-11954.txt.

Introduction

Sehyuk Park, Soyeon Caren Han, and Eduard Hovy present UniCast, a frame that adds text and image context to time series forecasts in two thousand twenty-five. Foundation models for time series are strong but read numbers alone, with fixed prompts. Real forecasts often hinge on context that numbers miss.

UniCast builds a prompt tuned to each case from the series plus vision and text inputs, through a transformer that draws out the shared gist. A routing step weighs how much each extra source should count right now, boosting the useful and muting the noisy. The base model stays frozen, so its broad skill is kept. Tests across many benchmarks beat all past foundation baselines.

Catalog record

CREATORSehyuk Park, Soyeon Caren Han, Eduard Hovy
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE46 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2508-11954

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

UniCast: A Unified Multimodal Prompting Framework for Time Series Forecasting Sehyuk Park, Soyeon Caren Han, Eduard Hovy Abstract Time series forecasting is a foundational task across domains, such as finance, healthcare, and environmental monitoring. While recent advances in Time Series Foundation Models (TSFMs) have demonstrated strong generalisation through large-scale pretraining, existing models operate predominantly in a unimodal setting, ignoring the rich multimodal context, such as visual and textual signals, that often accompanies time series data in real-world scenarios. This paper introduces a novel parameter-efficient multimodal framework, UniCast, that extends TSFMs to jointly leverage time series, vision, and text modalities for enhanced forecasting performance. Our method integrates modality-specific embeddings from pretrained Vision and Text Encoders with a frozen TSFM via soft prompt tuning, enabling efficient adaptation with minimal parameter updates. This design not only preserves the generalisation strength of the foundation model but also enables effective cross-modal interaction. Extensive experiments across diverse time-series forecasting benchmarks demonstrate that UniCast consistently and significantly outperforms all existing TSFM baselines. The findings highlight the critical role of multimodal context in advancing the next generation of general-purpose time series forecasters. Introduction Time series forecasting underpins critical decision-making across domains such as finance, healthcare, and climate science. Traditional statistical models like ARIMA ( Box et al. 2015 ) , and even more flexible machine learning approaches such as XGBoost ( Chen and Guestrin 2016 ) , offer limited capacity to capture non-linear and long-range temporal dependencies without extensive feature engineering ( Chen et al. 2023 ; Siami-Namini, Tavakoli, and Namin 2018 ) . Transformer-based models ( Vaswani et al. 2017 ) and pretrained Large Language Models (LLMs) ( Brown et al. 2020 ) have partially addressed these limitations, but their application to time series remains constrained by modality misalignment and computational inefficiency. Recent advances in Time Series Foundation Models (TSFMs) ( Ansari et al. 2024 ; Liu et al. 2024b ; Ekambaram et al. 2024 ) have shown promise by leveraging large-scale pretraining on diverse time series datasets to achieve strong generalization across forecasting tasks. However, current TSFMs operate in a unimodal regime, treating time series as isolated numerical signals while overlooking rich auxiliary modalities, such as visual observations and textual annotations, that often co-occur in real-world settings. We argue that this unimodal focus is a fundamental limitation. In practice, multimodal context provides critical complementary signals: visual data may capture environmental conditions affecting sensor readings, while textual metadata encodes domain-specific semantics. Incorporating such modalities can disambiguate temporal patterns, improve predictive robustness, and enhance generalization to unseen settings. Moreover, multimodal learning aligns with cognitive theories of human forecasting, which integrates multiple perceptual and semantic cues. In this paper, we propose UniCast, a novel parameter-efficient multimodal forecasting framework that extends TSFMs to jointly model time series, visual, and textual modalities. Our approach leverages pretrained modality-specific encoders, a Vision Encoder for visual data and a Text Encoder for textual data, to generate modality-specific embeddings. These embeddings are then integrated with a pretrained, frozen TSFM through a lightweight soft prompt tuning mechanism, allowing the model to adapt to multimodal inputs while updating only a small number of parameters. Specifically, visual and textual contextual information is first encoded via their respective pretrained encoders, producing dense representation vectors. These embeddings, along with the original time series data, are input to the TSFM, which performs the forecasting task by leveraging cross-modal interactions. By employing soft prompts rather than full fine-tuning, our UniCast preserves the generalization capabilities of the pretrained TSFM while significantly reducing the computational cost and risk of overfitting. Our main contributions are summarized as follows: • We introduce a theoretically grounded and practically motivated multimodal (vision and language) extension to Time Series Foundation Models. • We propose a multimodal soft prompt tuning strategy that enables parameter-efficient adaptation to heterogeneous modalities without retraining the entire time-series foundation model. • We provide extensive empirical evidence showing that multimodal context significantly improves forecasting accuracy and robustness across a wide range of time-series forecasting tasks. Related Work Pretrained Models for Time Series Forecasting. Inspired by the success of large language models (LLMs) in natural language processing, recent research has explored repurposing these models for time series forecasting. Early methods, such as LSTPrompt ( Liu et al. 2024a ) and TEMPO ( Cao et al. 2023 ) , leveraged prompting and lightweight adaptation techniques. Others, including Time-LLM ( Jin et al. 2023 ) , S^{2} IP-LLM ( Pan et al. 2024 ) , and TEST ( Sun et al. 2023 ) , proposed translating time series into tokenized patches or embedding-aligned sequences, enabling LLMs to process temporal signals as if they were linguistic inputs. Although these approaches inherit the generalisation capabilities of LLMs, they suffer from a fundamental modality mismatch: time series data is continuous, structured, and often multivariate, unlike natural language. This conversion not only complicates interpretability but may also degrade performance due to the loss of domain-specific inductive biases. To address these limitations, the research community has shifted toward Time Series Foundation Models (TSFMs), domain-specialised architectures pretrained directly on large-scale temporal data. MOMENT ( Goswami et al. 2024 ) introduced the Time Series Pile and proposed a pretraining pipeline that supports rapid downstream adaptation. Moirai ( Woo et al. 2024 ) introduced the LOTSA dataset and developed a model capable of handling variable-length sequences and mixed frequencies. Chronos ( Ansari et al. 2024 ) and Timer ( Liu et al. 2024b ) demonstrated how transformer-based architectures can be effectively adapted for temporal patterns. TimesFM ( Das et al. 2024 ) and TTM ( Ekambaram et al. 2024 ) have further demonstrated that decoder-only architectures can achieve competitive results with lower computational costs. Together, these models establish TSFMs as a promising direction, offering strong generalisation, scalability, and improved alignment with temporal structures. Building upon this foundation, our work aims to extend TSFMs beyond unimodal design by incorporating rich, complementary multimodal context. Multimodal Learning. Multimodal learning integrates heterogeneous data types, such as images, text, audio, and structured signals, into a unified representation framework ( Baltrušaitis, Ahuja, and Morency 2018 ) . This paradigm has been highly successful across tasks in vision-language modeling ( Radford et al. 2021 ; Alayrac et al. 2022 ) , video understanding ( Zhang, Li, and Bing 2023 ) , and embodied AI. For instance, LLaVA ( Liu et al. 2023 ) showed that freezing a vision encoder and an LLM while tuning a small projection layer enables effective vision-language alignment. M2PT ( Wang et al. 2024 ) extended this idea by introducing soft prompts for each modality and connecting them through a learnable interaction module, achieving strong performance with minimal tuning. While such strategies have led to impressive gains in multimodal NLP and vision, they remain largely unexplored in the context of time series forecasting. Most existing time-series models treat temporal signals in isolation, neglecting the auxiliary information, such as environmental visuals, sensor descriptions, and metadata that often accompanies real-world temporal data. This omission can lead to missed opportunities for richer contextual understanding, more robust predictions, and improved generalisability. Our Contribution. To bridge this gap, we propose the first parameter-efficient multimodal framework for time series forecasting that builds on pretrained TSFMs. Our approach integrates time series with auxiliary visual and textual information

Read the complete text