Noise Titration: Exact Distributional Benchmarking for Probabilistic Time Series Forecasting
Qilin Wang · 2026
Read Noise Titration: Exact Distributional Benchmarking for Probabilistic Time Series Forecasting by Qilin Wang (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-22219.txt.
Introduction
Wang proposes a new way to benchmark probabilistic time series forecasts. Standard tests watch one past trajectory, so claims about robustness stay untestable. His method adds measured Gaussian noise to known chaotic and stochastic systems. Since the true process and noise level are known, scoring can use exact likelihoods. He also extends Fern to output full covariance structures. Under this test, big zero-shot foundation models parrot context and fail under regime shifts. Fern keeps sharp calibration where those models collapse.
Catalog record
| CREATOR | Qilin Wang |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 59624 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.22219
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Noise Titration: Exact Distributional Benchmarking for Probabilistic Time Series Forecasting Qilin Wang Abstract Modern time series forecasting is evaluated almost entirely through passive observation of single historical trajectories, rendering claims about a model’s robustness to non-stationarity fundamentally unfalsifiable. We propose a paradigm shift toward interventionist, exact-statistical benchmarking. By systematically titrating calibrated Gaussian observation noise into known chaotic and stochastic dynamical systems, we transform forecasting from a black-box sequence matching game into an exact distributional inference task. Because the underlying data-generating process and noise variance are mathematically explicit, evaluation can rely on exact negative log-likelihoods and calibrated distributional tests rather than heuristic approximations. To fully leverage this framework, we extend the Fern architecture into a probabilistic generative model that natively parameterizes the Symmetric Positive Definite (SPD) cone, outputting calibrated joint covariance structures without the computational bottleneck of generic Jacobian modeling. Under this rigorous evaluation, we find that state-of-the-art zero-shot foundation models behave consistently with the context-parroting mechanism, failing systematically under non-stationary regime shifts and elevated noise. In contrast, Fern explicitly captures the invariant measure and multivariate geometry of the underlying dynamics, maintaining structural fidelity and statistically sharp calibration precisely where massive sequence-matching models collapse. 1 Introduction The Federal Reserve publishes the dot plot quarterly as part of its summary of economic projections : the dot plot chart carries projection of interest rates for the current and the next three calendar years, and for "longer run" (supposedly when the economy reaches a so-called stable state ). In real life, people pay little attention to those long-run forecasts. Instead, they outline different possible geopolitical scenarios and run forecasts under different scenarios. In other words, they actively model the different non-stationary shocks and trust their forecaster to function normally in different scenarios. Modern time series forecasting (TSF) is evaluated almost entirely in a passive, observational regime: models ingest a historical context and extrapolate a single realized future, while the underlying data-generating process (DGP) and the semantics of non-stationarity remain implicit. This makes robustness claims difficult to falsify: a “hard” dataset may reflect sensor artifacts, early-stopping quirks, or a single idiosyncratic realization rather than principled distribution shift. Consequently, we do not know whether state-of-the-art (SOTA) time series forecasters handle non-stationarity; we know only that they score well on benchmarks where non-stationarity happened to look a certain way. In experimental science, claiming a mechanism works requires specifying the intervention, controlling for confounds, and reporting calibrated statistical tests. Yet the dominant paradigm in TSF forecloses the very tools that make such rigor possible. 13 took a first step by arguing for intervention : we use synthetic datasets whose data-generating process (DGP) is a known dynamical system, and we treat the type, timing, and magnitude of non-stationary shocks as explicit experimental variables : a researcher then perturb either (1) parameters (2) state variables, or (3) switch regimes at controlled times. We observe if a method preserves predictive structure under counterfactuals (a range of varied paths) rather than chasing one single historical path. This interventionist view turns non-stationarity handling into a falsifiable, reproducible modelling task of the forecaster. As 13 puts it, this turns non-stationary shocks into an active modelling component, similar to making software tests for TSF models : we outline potential edge cases, like we outline the geopolitical possibilities, and prove , with full DGP and interventions knowledge, that the model functions properly under these cases. We actively discount the value of ultra long single trajectory prediction, as we know shocks, chaos and stochastic noise eventually make such long shot pointwise forecast impossible; we also actively discount the value of what 13 termed historical emulation , knowing well that different news, market obsessions etc could have resulted in a different trajectory; any model replicating the prediction either reach it by chance, or is overfit to that particular set of realized shocks. A Statistical Approach to Prediction: The Noise Titration Protocol Controlled DGPs make shock-handling claims falsifiable ; in this paper we propose a major expansion in this line of thinking: we make them inferrable . By wrapping calibrated Gaussian noise of known variance \sigma^{2} around deterministic trajectories, which can be understood as some simulated post-hoc measurement noises, the target distribution is \mathcal{N}(\mu_{\text{law}},\,\sigma^{2}I) by construction. Treating injected noise variance \sigma_{\text{inj}}^{2} as an experimental dial, we gain two complementary, falsifiable perspectives on model capacity: • Dialing Down (The Resolution Limit): As the injected noise decreases ( \sigma\to 0 ), we measure the model’s epistemic precision . By systematically unmasking the architecture’s inherent structural error, we quantify its exact lower bound : if the forecaster’s residuals pass a distributional goodness-of-fit test at an injected noise level of 10^{-3} but fail at 10^{-4} , we explicitly establish the model’s statistical resolution limit at 10^{-3} . • Dialing Up (The Robustness Threshold): As we increase the noise level, we test the model’s ability to extract the true invariant measure from severe aleatoric corruption . This identifies the exact Signal-to-Noise Ratio (SNR) threshold where the model’s manifold learning collapses into mean-guessing. We are not forcing the object to fit a statistical hammer; in contrast, now the ‘wind-tunnel’ interventionist experiments speaks the statistical inference language . SOTA MSE competition on a single historical realization is replaced by model performance in precise scientific terms : under this nonstationarity pattern, for this dynamical system family, the model retains accuracy across \sigma\in\{0.25,1.0\} and degrades at \sigma=2.0 . A single evaluation loop describes the non-stationarity we care, the noise structure (which can be heteroskedastic by design) we care, and sigma resolution level we care. We give models a rigorous, statistically verified “factory stamp” of their robustness profile before they ever leave the laboratory. This connects prediction back to its statistical root as a tool for decision-making , not an oracle that produces unqualified point forecasts, and forces the researcher to model the type of domain-specific non-stationarity for practical deployment. The Fern model from [ 13 ] serves as the main model due to its unique properties : it natively outputs a full Gaussian belief \mathcal{N}(\mu_{\theta},\Sigma_{\theta}) as well as the eigenvalues and eigenvectors of the covariance matrix. The parameterization makes closed-form W_{2} distances, exact coverage tests and calibrated hypothesis testing much easier. In this paper, we take three steps toward this paradigm: 1. We extend Fern with a multivariate Gaussian NLL objective, promoting its spectral parameterization from an implicit geometric belief to a calibrated probabilistic forecaster that natively outputs full covariance structure. 2. We benchmark Fern against foundation models (Chronos Bolt and Chronos 2 [ 2 , 1 ] , TimesFM 2.5 [ 5 ] ) across chaotic and nonstationary systems, demonstrating order-of-magnitude improvements on chaotic benchmarks. 3. We conduct noise titration experiments that characterize each model’s resolution profile, providing a falsifiable, quantitative alternative to single-realization leaderboard ranking. 2 Method Quick intro to Fern We briefly recap the central idea of the Fern model proposed by 13 : it views long-term time-series forecasting as conditional manifold transport : given a context window x_{1:L} (an evenly sampled time-delay embedding), predict a distribution over a future horizon y_{1:H} by transporting probability mass along the system’s latent geometry. Real systems blur or disrupt this geometry through stochastic noise, chaotic forces, and regime-switching non-stationary shocks. Fern addresses this by directly modelling a instance-wise