Information Criteria Fail for Dynamical Systems: Sampling Rate and Dimension Dependence

Kumar Utkarsh, Daniel M. Abrams, · 2025

Read Information Criteria Fail for Dynamical Systems: Sampling Rate and Dimension Dependence by Kumar Utkarsh, Daniel M. Abrams, (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-14931.txt.

Introduction

Kumar Utkarsh and Daniel M. Abrams show that two favorite tools for choosing between models can quietly fail on data from dynamical systems. The Akaike and Bayesian information criteria balance fit against complexity, and scientists use them everywhere. Their math, however, assumes each data point is independent, which time-ordered data from a moving system never is.

Testing simple systems, from steady decay to a swinging oscillator to chaos, the authors find the verdicts swing with the sampling rate and the system's dimension. They derive formulas that predict when the criteria will mislead. Those formulas let experimentalists pick sampling plans that keep model selection honest.

Catalog record

CREATORKumar Utkarsh, Daniel M. Abrams,
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE25 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2511-14931

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Information Criteria Fail for Dynamical Systems: Sampling Rate and Dimension Dependence Kumar Utkarsh, Daniel M. Abrams, Abstract Information criteria such as Akaike’s (AIC) and Bayes’ (BIC) are widely used for model selection in physics and beyond, quantifying the tradeoff between model complexity and goodness-of-fit to enforce parsimony. However, their derivation assumes uncorrelated samples, an assumption systematically violated by dynamical systems data. Here, through analysis of simple but representative dynamical models—exponential decay, harmonic oscillation, and chaos—we demonstrate that model selection depends sensitively on sampling rate and system dimensionality. We derive explicit formulas predicting when standard information criteria fail that should be adaptable to many real-world scenarios, enabling experimentalists to design sampling protocols that avoid pathological regimes. I Introduction Information criteria, especially the Akaike Information Criterion (AIC) 2 and Bayesian Information Criterion (BIC) 19 , are fundamental tools for model selection in many fields including physics, statistics, and machine learning 4 ; 13 ; 28 ; 5 . Their theoretical foundation lies in approximating the Kullback-Leibler (KL) divergence 14 between the true data-generating process and the candidate models, and, in a deeper sense, in information theory’s connection to mathematical models 24 ; 25 ; 10 . These criteria have found extensive application across physics (see, e.g., 26 ; 17 ; 3 ; 27 ; 8 ) and, indeed across all of quantitative science (see, e.g., 11 ; 18 ; 1 . However, their application to dynamical systems with temporal correlations has received limited theoretical scrutiny. While it is well established that information criteria assume independent observations 5 —an assumption usually violated by dynamical systems data—a tractable predictive framework for experimental design remains absent. Existing approaches either restrict themselves to a particular class of models 12 ; 9 or require computationally intensive simulation-based methods 20 ; 23 . What is missing are explicit scaling laws that enable experimentalists to understand, a priori , which sampling protocols will yield reliable model selection and which will lead to systematic failures. Here, we present a series of simple examples to illustrate two key problems: Sampling rate dependence. The model selected by an information criterion can depend sensitively on the sampling rate of the observations. This is counterintuitive, as one might expect more data (higher sampling rates) to always improve model selection. However, both under-sampling and over-sampling can lead to incorrect conclusions because information criteria assume independent observations 5 . This problem is particularly acute in physics, where experimenters often have significant control over sampling parameters but may unknowingly choose rates that lead to systematic model misidentification 29 ; 7 . Dimension dependence. The reliability of model selection depends on the system dimension. Surprisingly, this represents a “dimensionality blessing” rather than the usual curse—higher-dimensional systems with many independent realizations can overcome complexity penalties through collective evidence 19 ; 21 . However, this blessing depends critically on how the number of parameters scales with dimension, an issue that has received limited attention in the context of collective dynamical systems 25 . These pathologies are particularly concerning because they can lead to systematic misidentification of physical mechanisms. For instance, genuine relaxation dynamics might be dismissed as noise due to poor sampling choices, or tractable models for behavior might be overlooked in low-dimensional observations 6 . The implications extend beyond model selection to parameter estimation and uncertainty quantification 16 . Our analysis provides complete characterization for first-order linear systems and harmonic oscillators—ubiquitous dynamical motifs appearing throughout physics. The resulting scaling laws offer immediate practical guidance: experimentalists can evaluate whether their sampling protocols fall in reliable or pathological regimes, and adjust accordingly before data collection. Unlike previous studies that focus on comparing different information criteria (e.g., 1 ), we demonstrate that the problems are intrinsic to the application of likelihood-based methods to temporally correlated data, regardless of which specific criterion is used 1 1 1 We suspect that similar problems also exist in practice for non-likelihood methods like minimum description length (MDL). . Our goal is to establish a tractable predictive framework for experimental design to mitigate these problems. II Sampling Rate Dependence Equilibrating Systems: Consider a simple deterministic decay model governed by \dot{x}=-\lambda x, (1) where \lambda>0 is the decay rate and the overdot indicates a time derivative. Assume an initial condition x(0)=x_{0} and normally distributed observation noise \eta\sim\mathcal{N}(\mu,\sigma^{2}) , where \mu and \sigma^{2} are the noise mean and variance, respectively. Then the observations should satisfy x(t)\sim x_{0}e^{-\lambda t}+\eta . Fixed total data: Suppose we sample x(t) at a fixed sampling frequency f (i.e., at discrete times with interval \Delta t=1/f ), collecting M total data points. We wish to answer the key question: are these data more consistent with the decay model in Eq. ( 1 ), or are they more consistent with a null model that assumes pure noise? In the low sampling frequency limit f\to 0^{+} (large \Delta t ), the sampled data resemble independent Gaussian draws, leading to selection of the “pure noise” model. As sampling frequency grows, however, temporal correlations emerge and the decay model is eventually favored. In the high sampling frequency limit f\to\infty (small \Delta t ), however, the sampled data again resemble independent Gaussian draws, and once again the pure noise (null) model is selected. The crossovers between these different selection regimes depend on the decay rate \lambda , initial condition x_{0} , noise level \sigma , and number of samples M . For the low-frequency crossover, sparse sampling fails to capture sufficient decay structure when f\ll\lambda . In this regime, only the first \sim f/\lambda data points contain significant signal, and the signal variance relative to noise variance becomes too small to justify the extra complexity penalty. Detailed analysis (see Appendix A ) yields the lower crossover frequency: f_{c}^{(1)}=\frac{8\lambda\sigma^{2}}{x_{0}^{2}}. (2) For the high-frequency crossover, the observation window t_{\max}=M/f becomes too short to capture decay dynamics when f\gg\lambda . In this regime, the signal appears nearly constant over the sampling period, with variance scaling as x_{0}^{2}\Lambda^{2}/12 where \Lambda=\lambda M/f\ll 1 is the number of decay timescales observed. Balancing this against the AIC complexity penalty yields the upper crossover frequency: f_{c}^{(2)}=\frac{M^{3/2}\lambda x_{0}^{2}}{4\sqrt{3}\sigma}. (3) So for f_{c}^{(1)}\lesssim f\lesssim f_{c}^{(2)} we expect the “true” decay model to be selected, but for sampling frequencies much higher or lower we expect the noise model to be selected. Fig. 1 demonstrates numerically that this is indeed the result when using AIC. Figure 1: Equilibration: Selected model varies with sampling rate. Top: Comparison of AIC values for true deterministic decay model (blue) and pure noise null model (red) as a function of sampling frequency f . Theoretical crossover points are indicated by vertical orange dashed lines: f_{c}^{(1)}=8\lambda\sigma^{2}/x_{0}^{2}=5.0 and f_{c}^{(2)}=M^{3/2}\lambda x_{0}^{2}/(4\sqrt{3}\sigma)\approx 2000 . Bottom: Proportion of trials in which each model is selected as a function of sampling frequency f (1000 trials). Note that the noise model is selected at high and low frequencies (left and right on graph). Parameters: (x_{0},\lambda,\mu,\sigma,M)=(1,0.1,0,2.5,2000) . Points in upper panel represent means over trials at each frequency. This analysis clearly demonstrates the counter-intuitive result that both under-sampling and over-sampling can lead to incorrect model selection, even when the true model is included in the candidate set. This suggests that optimal experimental design for dynamical systems must carefully consider intrinsic time scales of the system

Read the complete text