A Set-Sequence Model for Time Series
Elliot L. Epstein, Apaar Sadhwani, Kay Giesecke · 2025
Read A Set-Sequence Model for Time Series by Elliot L. Epstein, Apaar Sadhwani, Kay Giesecke (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-11243.txt.
Introduction
Elliot Epstein, Apaar Sadhwani, and Kay Giesecke propose a model for forecasting many related time series at once in a 2025 paper. Loans, stocks, and customers each follow their own history, yet shared forces move them together. Standard sequence models read one series at a time and need hand-built summaries to sense the group.
Their Set-Sequence model adds a set module that summarizes the whole group at each time step. A sequence module then forecasts each unit using its own features plus that summary. The design handles uneven series and changing group sizes, and scales in a straight line with group size. It beat strong baselines on stock portfolios and loan risk tasks.
Catalog record
| CREATOR | Elliot L. Epstein, Apaar Sadhwani, Kay Giesecke |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 72 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-11243
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
A Set-Sequence Model for Time Series Elliot L. Epstein, Apaar Sadhwani, Kay Giesecke Abstract Many prediction problems across science and engineering, especially in finance and economics, involve large cross-sections of individual time series, where each unit (e.g., a loan, stock, or customer) is driven by unit-level features and latent cross-sectional dynamics. While sequence models have advanced per-unit temporal prediction, capturing cross-sectional effects often still relies on hand-crafted summary features. We propose Set-Sequence , a model that learns cross-sectional structure directly, enhancing expressivity and eliminating manual feature engineering. At each time step, a permutation-invariant Set module summarizes the unit set; a Sequence module then models each unit’s dynamics conditioned on both its features and the learned summary. The architecture accommodates unaligned series, supports varying numbers of units at inference, integrates with standard sequence backbones (e.g., Transformers), and scales linearly in cross-sectional size. Across a synthetic contagion task and two large-scale real-world applications—equity portfolio optimization and loan risk prediction—Set-Sequence significantly outperforms strong baselines, delivering higher Sharpe ratios, improved AUCs, and interpretable cross-sectional summaries. 1 Introduction Many key problems in finance—such as constructing stock portfolios or estimating risk for a pool of loans—involve predicting outcomes for large populations of correlated units. Similar problems appear in science and engineering where the individual units may be sensors, devices, or customers. A key characteristic of such problems is they involve a varying number of correlated units, with each unit represented by a time series of dynamic features. In short, they involve a set of sequences where the sequences share the time axis. This poses a two-dimensional challenge: capturing dependencies across the cross-sectional dimension of M units, while modeling the temporal dynamics over T time steps. Directly modeling the joint distribution of the M units is intractable due to its size. For example, a portfolio of M=10,000 loans where each loan has 50 features per time step, results in a (joint) time series with 50,000 features per time step ( Anenberg & Kung (2014) ; Azizpour et al. (2018) ). To avoid this, existing approaches make predictions for each unit separately, while augmenting the unit’s own features with hand-crafted summaries for the cross-section; the model learns the temporal dynamics but the cross-sectional features are hand-designed. For example, Sadhwani et al. (2020) employs aggregate foreclosure rate as a summary feature across the loans. This approach requires domain expertise and is unlikely to capture all latent effects. To address this two-dimensional challenge, we propose the Set-Sequence model, which decouples the modeling of cross-sectional and temporal dependencies by learning latent cross-sectional effects directly from data. The architecture consists of two components (see Figure 1 ). First, a Set module processes the permutation-invariant cross-section at each time step, leveraging unit exchangeability 1 1 1 Exchangeability implies that unit identities (e.g., stock tickers) are irrelevant—behavior is determined solely by observed features. Formally, for a problem of size M , the unit-level feature vectors X^{i} are exchangeable: P(X^{1},\dots,X^{M})=P(X^{\pi(1)},\dots,X^{\pi(M)}) for any permutation \pi . to compute an order-invariant summary of the population state. Second, this summary is concatenated to each unit’s features and passed to a Sequence module (e.g., Transformer or RNN) that models temporal dynamics. The Set-Sequence model fulfills three key desiderata: inference over a variable number of units, handling unaligned units with different start and end times, and integration with standard sequence models. This simple architecture yields significant practical benefits: it removes the need for handcrafted features and seamlessly incorporates cross-sectional dependencies into unit-level predictions. Figure 1: The Set-Sequence model. The Set model estimates a cross-sectional summary at each time, in linear complexity over the number of units. This augments the original features of each unit and the Sequence model consumes this augmented series to make predictions for each unit independently. Note that the Set model has a look-back of L time periods, where L\geq 1 is a model parameter. The number of units M and time periods T may vary at inference. Theoretical analysis highlights key properties of the Set-Sequence model with direct practical implications. First, a forward-pass complexity result shows that mean-based cross-sectional aggregation scales linearly in the number of units M , in contrast to the quadratic scaling of attention-based methods—enabling efficient training on large cross-sections. Second, under exchangeability, a polynomial sufficiency result guarantees that pooled monomials up to degree k can approximate any continuous permutation-invariant cross-sectional statistic. Together, these results motivate the use of simple set summaries as a scalable and expressive default, capable of capturing rich latent structure without incurring prohibitive computational cost. We first benchmark the performance and efficiency of the Set-Sequence model on a synthetic contagion task that mimics default prediction for a large pool of exchangeable loans. Defaults are driven by a shared latent factor \lambda_{t} , capturing contagion effects observed in corporate and consumer credit markets. The Set-Sequence model substantially outperforms state-of-the-art sequence models, achieving near-optimal performance across a wide range of pool sizes, with accuracy improving as coverage increases. It also exhibits strong interpretability: the learned set summaries closely track the true latent factor ( \rho=0.95 ). Notably, the simple linear aggregation used in the Set module outperforms more complex multi-head attention layers, both in accuracy and efficiency. We next evaluate the model on a real-world stock portfolio construction task using return data for 36,600 U.S. equities. The goal is to select daily portfolio weights that maximize the annualized Sharpe ratio (a widely used performance criterion). We compare Set-Sequence against leading sequence models and the task-specific CNN-Transformer of Guijarro-Ordonez et al. (2022) . Over a 15-year out-of-sample period, Set-Sequence achieves 22–42% higher Sharpe ratios than the strongest baselines. These improvements translate into economically meaningful investment gains. Finally, we apply Set-Sequence to a large-scale U.S. mortgage risk prediction task, using 5 million loan-month observations and 52 dynamic loan-level and macroeconomic features. The task involves classifying the next-month mortgage state (e.g., current, 30/60/90 days delinquent, foreclosure, prepayment). By effectively capturing latent cross-unit dependencies, Set-Sequence achieves significantly higher AUCs than both general-purpose sequence models and the domain-specific deep learning benchmark of Sadhwani et al. (2020) . These gains translate into more accurate risk metrics and improved decision support for large-scale mortgage portfolio management. This work highlights the value of exploiting exchangeability in high-dimensional time series and demonstrates that the Set-Sequence model offers a practical and effective solution. It: (1) Outperforms strong baselines on both synthetic and real-world tasks; (2) Scales efficiently to large cross-sections, with linear complexity in the number of units and support for a variable number of units at inference; and (3) Provides interpretability , with low-dimensional set summaries that capture and reveal latent cross-sectional structure. Together, these properties make Set-Sequence a compelling architecture for modern time-series prediction problems involving large populations of interacting units. 2 Related Literature Traditional methods for modeling multiple time series include Vector Autoregressive (VAR) models Sims (1980) , which scale quadratically in the cross-sectional dimension. Linear factor models Chamberlain & Rothschild (1983) ; Stock & Watson (2002a) ; Stock & Watson (2011) reduce dimensionality but struggle with non-linear dependencies. Deep learning-based factor models Wang et al. (2019) address this limitation, yet require retraining when new units are introduced. Recent deep learning architectures for time series, though powerful, are typically not designed to exploit cross-sectional exchangeability as they rely