System-Anchored Knee Estimation for Low-Cost Context Window Selection in PDE Forecasting

Wenshuo Wang, Fan Zhang · 2026

Read System-Anchored Knee Estimation for Low-Cost Context Window Selection in PDE Forecasting by Wenshuo Wang, Fan Zhang (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-25025.txt.

Introduction

Wang and Zhang tackle a cost choice in physics simulators: how much past data each forecast step should see. Neural PDE simulators roll a field forward from a fixed window of past frames. Picking that window by full search is pricey, and theory-based picks may miss what helps rollouts. Their method, SAKE, first draws a small candidate set from physical traits of the system. It then picks the knee point by real rollout scores. Across eight PDEBench families, SAKE is the strongest cheap picker at matched budget.

Catalog record

CREATORWenshuo Wang, Fan Zhang
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE101237
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.25025

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

System-Anchored Knee Estimation for Low-Cost Context Window Selection in PDE Forecasting Wenshuo Wang, Fan Zhang Abstract Autoregressive neural PDE simulators predict the evolution of physical fields one step at a time from a finite history, but low-cost context-window selection for such simulators remains an unformalized problem. Existing approaches to context-window selection in time-series forecasting include exhaustive validation, direct low-cost search, and system-theoretic memory estimation, but they are either expensive, brittle, or not directly aligned with downstream rollout performance. We formalize explicit context-window selection for fixed-window autoregressive neural PDE simulators as an independent low-cost algorithmic problem, and propose System-Anchored Knee Estimation (SAKE) , a two-stage method that first identifies a small structured candidate set from physically interpretable system anchors and then performs knee-aware downstream selection within it. Across all eight PDEBench families evaluated under the shared L ∈ { 1 , … , 16 } L\in\{1,\dots,16\} protocol, SAKE is the strongest overall matched-budget low-cost selector among the evaluated methods, achieving 67.8% Exact, 91.7% Within-1, 6.1% mean regret@knee, and a cost ratio of 0.051 (94.9% normalized search-cost savings). 1 Introduction Autoregressive neural simulators for PDEs predict the evolution of physical fields by advancing one time step at a time from a finite history of past states [ 1 , 2 ] . In this setting, choosing the context window length is a fundamental yet surprisingly under-studied problem: the window determines how much temporal information the simulator receives, but also how much training, evaluation, and tuning cost must be paid. Existing work typically fixes this window as a task-specific hyperparameter or a priori [ 3 , 1 ] , or folds it implicitly into memory-based architectures [ 4 ] . Other work examines its effect only through ablations, rather than treating context-window selection itself as an independent optimization problem [ 5 , 6 ] . As a result, a basic question remains open: can we identify the performance-critical window for a given autoregressive neural simulator without paying the cost of exhaustive search? The problem is challenging because the best window depends jointly on the underlying system dynamics, the simulator architecture, and the downstream rollout objective. More broadly, context-window selection has been studied in time-series forecasting, where existing approaches can be roughly grouped into three families [ 11 ] . The first is exhaustive validation over candidate windows [ 11 ] . The second is direct low-cost search, which replaces full evaluation with cheap pilots, early stopping, or multi-fidelity resource allocation over a generic shortlist of candidate windows [ 12 , 13 , 17 , 18 , 19 ] . The third is system-theoretic memory estimation, which infers a characteristic memory or embedding scale from the underlying dynamics and uses it to choose or bracket plausible windows [ 14 , 15 , 20 , 16 ] . Yet none of them fully addresses our setting. Exhaustive validation is accurate but expensive [ 11 ] ; direct search is cheaper but often system-agnostic and brittle [ 12 , 13 ] ; and memory estimates are physically meaningful but do not directly determine the window that optimizes downstream rollout performance in neural PDE simulators [ 14 , 15 , 16 ] . We therefore formalize explicit context-window selection for autoregressive neural PDE simulators as an independent low-cost algorithmic problem, and propose System-Anchored Knee Estimation (SAKE) . SAKE is a two-stage method. It first extracts physically interpretable system anchors from clean-system dynamics to identify a small but structured region of plausible windows. It then performs knee-aware downstream selection only within this anchor-induced candidate set, using a small number of task-level evaluations to choose the final window. By coupling system priors with lightweight downstream selection, SAKE avoids both the cost of exhaustive sweeps and the brittleness of generic low-cost search, providing a more reliable and efficient way to determine performance-critical context windows. We evaluate SAKE on two representative PDEBench tasks, four autoregressive simulator backbones, and three training seeds in the main body, and complement these case-level results with an appendix-wide aggregate over all eight PDEBench families under a shared L\in\{1,\dots,16\} protocol. Across this fixed-window setting, we measure exact oracle-knee recovery, absolute window error, regret at the knee, and search cost. The aggregate results show that SAKE has the strongest overall matched-budget profile among the compared selectors in this scope: it achieves 67.8% Exact, 91.7% Within-1, 6.1% mean regret@knee, and a 0.051 cost ratio, outperforming the generic shortlist and ASHA baselines without using a larger search budget. These gains do not come from more aggressive search. Instead, they come from coupling system-informed anchors with lightweight downstream selection. In summary, the contributions of this paper are: • We are the first to formalize explicit context-window selection for autoregressive neural PDE simulators as an independent low-cost algorithmic problem. This problem is important because exhaustive context-window tuning can be computationally expensive in practice. • We propose SAKE, a two-stage method that combines physically interpretable system anchors with knee-aware downstream selection. • We empirically validate the effectiveness of SAKE across multiple PDE datasets, simulator backbones, and training seeds. In the fixed-window autoregressive setting studied here, SAKE provides a more reliable and efficient low-cost selector of context windows than the generic shortlist baselines considered in the main text. 2 Related Work 2.1 Temporal dependence and historical information in neural simulators Recent work on time-dependent neural PDE solvers increasingly treats temporal dependence as a central challenge rather than a secondary implementation detail. Standard neural operators have been found to struggle with temporal dynamics beyond the horizons seen during training, architecture papers now target time-evolving dynamics directly, long-rollout studies make stable temporal prediction a primary concern, and recent benchmarks explicitly emphasize rollout metrics for temporal generalization [ 8 , 9 , 6 , 10 ] . This importance of historical information is made especially explicit by On the Benefits of Memory for Modeling Time-Dependent PDEs , which points out that many existing approaches are effectively Markovian and shows that using past states can substantially improve performance when the system is partially observed, noisy, or rich in high-frequency content [ 4 ] . Yet, despite this growing recognition, historical information is still usually handled implicitly rather than selected explicitly: many methods condition on only the current state or immediate past state, while others absorb memory into a dedicated architecture instead of treating the amount of accessible history as a separate modeling decision [ 4 , 8 ] . When the role of history is examined more directly, it is typically through architecture ablations, rollout-strategy studies, or benchmark analyses aimed at temporal extrapolation and stability, rather than through a procedure that explicitly selects a context window for a fixed autoregressive neural simulator [ 8 , 6 , 10 ] . The gap, therefore, is not that prior work ignores temporal context; rather, it has not isolated explicit context-window selection as a standalone algorithmic problem. Existing work usually fixes history as part of the modeling setup, bakes it into a memory-aware architecture, or studies it only through ablations and rollout analyses, whereas our work makes the choice of context window itself the object of study [ 4 , 8 , 6 , 10 ] . In particular, the present paper focuses on fixed-window one-step autoregressive simulators, treating adaptive-memory or

Read the complete text