DELPHYNE: A Pre-Trained Model for General and Financial Time Series

Xueying Ding, Aakriti Mittal, Achintya Gopal · 2025

Read DELPHYNE: A Pre-Trained Model for General and Financial Time Series by Xueying Ding, Aakriti Mittal, Achintya Gopal (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-06288.txt.

Introduction

Xueying Ding, Aakriti Mittal, and Achintya Gopal introduced a pre-trained model for financial time series, in a 2025 paper. General time series models have spread fast, yet on finance tasks they fail to beat simple finance baselines. The team blames two causes: too little finance data in pre-training, and harmful transfer from series whose patterns clash with market data.

Their model, Delphyne, is pre-trained with finance in the mix from the start. Series data are messy in ways text is not: they run continuous, noisy, and at mixed speeds and lags. On public data sets, Delphyne matched strong foundation and full-shot models with only a few tuning steps. It also led on a range of finance tasks. The authors offer it as proof that domain data belong inside pre-training, not just at the end.

Catalog record

CREATORXueying Ding, Aakriti Mittal, Achintya Gopal
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE105 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2506-06288

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

DELPHYNE: A Pre-Trained Model for General and Financial Time Series Thanks: Work done while employed at Bloomberg. Xueying Ding, Aakriti Mittal, Achintya Gopal Abstract Time-series data is a vital modality within data science communities. This is particularly valuable in financial applications, where it helps in detecting patterns, understanding market behavior, and making informed decisions based on historical data. Recent advances in language modeling have led to the rise of time-series pre-trained models that are trained on vast collections of datasets and applied to diverse tasks across financial domains. However, across financial applications, existing time-series pre-trained models have not shown boosts in performance over simple finance benchmarks in both zero-shot and fine-tuning settings. This phenomenon occurs because of a i) lack of financial data within the pre-training stage, and ii) the negative transfer effect due to inherently different time-series patterns across domains. Furthermore, time-series data is continuous, noisy, and can be collected at varying frequencies and with varying lags across different variables, making this data more challenging to model than languages. To address the above problems, we introduce a Pre-trained Mo DEL for FIN ance Tim E -series ( Delphyne ). Delphyne achieves competitive performance to existing foundation and full-shot models with few fine-tuning steps on publicly available datasets, and also shows superior performances on various financial tasks. 1 Introduction Time series is one of the most ubiquitous modalities in finance. Time-series analysis is critical to various tasks, such as asset pricing, volatility modeling, risk management, economic indicator analysis, etc. Traditional statistical methods for financial time series include Autoregressive Integrated Moving Average (ARIMA), Generalized Autoregressive Conditional Heteroskedasticity (GARCH), and Vector Autoregressive (VAR) models. In recent years, deep learning-based methods are being applied to these financial tasks (e.g., Horvath et al. 2019 ; Araujo & Gaglianone 2023 ; Liu et al. 2024e ; Gopal 2024 ). Following the surge of large language models (LLMs), research has explored the possibility of large deep-learning models for time-series analysis. One approach focuses on prompting and adapting existing pre-trained language models for time-series tasks ( Zhou et al. 2023 ; Jin et al. 2024 ; Chang et al. 2023 ; Gruver et al. 2023 ) , while another line of research trains foundation models directly on time-series data ( Woo et al. 2024 ; Das et al. 2024 ; Goswami et al. 2024 ) . Both approaches aim to train a single model with large-scale cross-domain data that can be applied to various tasks, as opposed to full-shot methods that require training a distinct model for each dataset. However, previous research indicates that directly prompting LLMs for financial tasks brings only modest benefits; moreover, most LLM-based research focuses only on forecasting directional trends of stock returns, offering fewer practical applications than traditional methods ( Chen et al. 2023 ; Yu et al. 2023 ; Lopez-Lira & Tang 2023 ) . Preliminary experiments also reveal that pre-trained models do not demonstrate significant improvement over GARCH in risk analysis (Sec. 5 ). One reason for this limitation is that the existing public datasets for pre-training lack sufficient financial data to learn patterns and distributional properties that are specific to time-series data in finance. Jointly training on financial data alongside public datasets presents an additional challenge: the substantial differences between domains can lead to negative transfer . Negative transfer has been extensively discussed within the literature, defined as a case when “transferring knowledge from the training data has an negative impact on the target tasks” ( Wang et al. 2019 ) . Negative transfer is identified as a key obstacle towards pre-training graph foundation model ( Wang et al. 2024a ; Mao et al. 2024 ) , yet is not explored within the context of time-series data. In Sec. H , we show that the negative transfer effect is a real challenge when pre-training time-series models. Cross-domain transfer learning is particularly difficult, as time-series data tends to be noisier and more continuous compared to languages and images. To alleviate the negative transfer effect, we believe that fine-tuning is the only remedy. We contend that the strength of pre-trained time-series models lies in their capacity to rapidly “unlearn” the biases of the pre-training stage and “adapt” to the specific distribution of new tasks, given limited training data and time. Building on our observation and analysis of negative transfer, we introduce our pre-trained time-series model, which incorporates several architectural modifications designed to better handle time-series data and tasks. Time-series data often involve an arbitrary number of variates, each collected at different frequencies (e.g., where quarterly profit growth is forecasted from daily or weekly sales data). Additionally, finance tasks often involve nowcasting, where some variates contain contemporaneous data. Our model is an encoder-based transformer that employs the any-variate attention mechanism, introduced by Woo et al. 2024 . In addition, we implement a missing data mask alongside the forecast mask, which enhances the model’s ability to adapt to cross-frequency training while minimizing negative interference ( Van Ness et al. 2023 ) . Probabilistic forecasting is an essential feature required for uncertainty quantification and anomaly detection within time series. Existing literature either assumes a fixed output distribution ( Salinas et al. 2020a ) or utilizes a mixture of several distributions ( Woo et al. 2024 ) . However, when the underlying data exhibits contrasting distributions and supports, a single distribution may be inadequate. Our model simplifies the output by using a mixture of Student-T distributions, which are widely used in finance for heavy-tailed tasks. In our preliminary studies, this approach shows comparable performance to Woo et al. 2024 on downstream tasks. In Sec. 3 , we introduce additional architectural modifications and present corresponding ablation studies. To this end, we train our Pre-Trained Mo DEL for FIN ance Tim E -series ( Delphyne ), the first time-series model capable of both general and finance-specific tasks. Delphyne is trained on both LOTSA (the largest public times-series dataset ( Woo et al. 2024 ) ) and financial data (Sec. 4 ). We conduct experiments for both general and finance time-series tasks, demonstrating that Delphyne consistently outperforms or matches state-of-the-art baselines. Our key contributions are as follows: • Demonstrate the presence of negative transfer in pre-trained time-series models, highlighting how this effect sets them apart from LLMs, emphasizing that the value of pre-trained time-series models lies in their fine-tuning capability, which mitigates the negative transfer effect and enhances downstream performance with minimal iterations. • Introduce several architectural modifications in our model and demonstrate their significance through comprehensive ablation studies. • Delphyne is the first time-series pre-trained model to excel in both general time-series tasks and a variety of downstream finance tasks. 2 Negative Transfer Effect Negative transfer has been widely studied in foundation models across different modalities ( Wang et al. 2024a ) . This problem is typically seen as the model’s reduced performance on downstream tasks due to mismatches between the source training data and the target distribution ( Wang et al. 2019 ) . However, this phenomenon has not been thoroughly explored in the context of time series. In pre-trained time-series models, negative transfer can occur when cross-domain data is added during pre-training. The appearances of data from too different distributions can lead to less effective

Read the complete text