The Illusion of Consistency: Selection-Induced Bias in Gated Kalman Innovation Statistics
Barak Or · 2025
Read The Illusion of Consistency: Selection-Induced Bias in Gated Kalman Innovation Statistics by Barak Or (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2512-18508.txt.
Introduction
Barak Or examines a quiet statistical trap inside Kalman filter tracking, the validation gate. Trackers only accept measurements whose normalized innovation squared falls below a threshold, discarding the rest as implausible. His paper, posted to arXiv in December 2025, shows this sensible filter changes the very statistics engineers later compute from the accepted data.
Under standard linear-Gaussian assumptions, he derives exact first and second moments of the innovation once ellipsoidal gating is applied. Gating shrinks the innovation covariance by a fixed, dimension-dependent amount. He then shows nearest-neighbor association acts as a second selection step with its own unavoidable energy contraction. Closed-form results for the two-dimensional case measure the combined effect. The lesson: after gating, nominal innovation statistics can never be fully preserved.
Catalog record
| CREATOR | Barak Or |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 38 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2512-18508
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
The Illusion of Consistency: Selection-Induced Bias in Gated Kalman Innovation Statistics Barak Or Abstract Validation gating is a fundamental component of classical Kalman-based tracking systems. Only measurements whose normalized innovation squared (NIS) falls below a prescribed threshold are considered for state update. While this procedure is statistically motivated by the chi-square distribution, it implicitly replaces the unconditional innovation process with a conditionally observed one, restricted to the validation event. This paper shows that innovation statistics computed after gating converge to gate-conditioned rather than nominal quantities. Under classical linear–Gaussian assumptions, we derive exact expressions for the first- and second-order moments of the innovation conditioned on ellipsoidal gating, and show that gating induces a deterministic, dimension-dependent contraction of the innovation covariance. The analysis is extended to NN association, which is shown to act as an additional statistical selection operator. We prove that selecting the minimum-norm innovation among multiple in-gate measurements introduces an unavoidable energy contraction, implying that nominal innovation statistics cannot be preserved under nontrivial gating and association. Closed-form results in the two-dimensional case quantify the combined effects and illustrate their practical significance. I Introduction Kalman-based tracking systems are a cornerstone of modern aerospace, radar, and navigation applications, where reliable state estimation is required under uncertainty. A central role in such systems is played by innovation-based statistics, which are used for measurement validation, data association, consistency monitoring, and adaptive tuning. Among these, the normalized innovation squared (NIS) is particularly attractive due to its simple statistical characterization under nominal linear Gaussian assumptions: in the absence of additional selection mechanisms, the NIS follows a chi-square distribution whose moments provide natural reference values for diagnostic and tuning procedures. In operational tracking systems, however, innovation statistics are rarely observed in their unconditional form. Prior to data association and state update, measurements are typically subjected to ellipsoidal validation gating, whereby only innovations whose Mahalanobis distance falls below a prescribed threshold are accepted. Validation gating is widely justified on statistical and practical grounds, as it limits computational complexity and suppresses gross outliers. As a result, nearly all innovation-based diagnostics used in practice operate on a post-gate innovation stream rather than on the nominal innovation process assumed in classical Kalman filter theory. Despite its ubiquity, the statistical consequences of validation gating are rarely treated explicitly. Once gating is applied, the innovations that enter association, filtering, and diagnostic logic are no longer samples from the nominal Gaussian distribution, but from a distribution truncated to the validation region. Consequently, innovation-based statistics computed after gating need not satisfy the classical reference properties associated with the chi-square law. In particular, empirical NIS statistics obtained from accepted measurements may systematically deviate from their nominal expectations even when the underlying system model and noise statistics are perfectly matched. A closely related issue arises in data association. When multiple measurements fall inside the validation gate, NN association is commonly employed to select a single measurement for the state update. While computationally efficient and widely used, NN association further conditions the observed innovation through order-statistic selection, as it selects the innovation with minimum normalized distance among multiple candidates. The combined effect of validation gating and NN association therefore induces a multi-stage selection process whose impact on innovation statistics is not captured by nominal Kalman filter assumptions. The objective of this paper is to isolate and characterize the statistical effects induced solely by validation gating and NN association, independent of clutter, false alarms, nonlinear dynamics, or modeling errors. Within the classical linear–Gaussian Kalman filtering framework, we provide an exact characterization of the innovation moments conditioned on ellipsoidal gating and show that validation gating induces a deterministic, dimension-dependent contraction of the innovation covariance. We further show that NN association acts as an additional statistical selection operator and introduces an unavoidable, multiplicity-dependent contraction of innovation energy through order-statistic selection. Together, these results imply that nominal innovation statistics cannot be preserved under nontrivial gating and association, even when all Kalman filter assumptions are satisfied. I-A Related Work Innovation-based statistics are a fundamental component of Kalman filtering theory and practice, where quantities such as the NIS are routinely used for consistency monitoring, measurement validation, and adaptive tuning. Under nominal linear Gaussian assumptions, the statistical properties of the innovation process are well understood and documented in classical estimation references [ 10 , 9 , 8 , 3 , 4 ] . In practical tracking systems, innovation statistics are commonly employed in adaptive filtering and noise covariance estimation schemes, often relying on assumed chi-square properties of the NIS [ 12 , 1 , 20 , 21 ] . Such approaches implicitly assume that the observed innovation sequence is representative of the nominal innovation distribution. However, in operational settings, innovation statistics are almost always evaluated after validation gating and data association, conditions under which these assumptions may no longer hold. Validation gating and NN association are standard components of tracking systems in aerospace, radar, and navigation applications [ 3 , 17 , 19 , 11 ] . Despite their widespread use, the statistical impact of these selection mechanisms on innovation-based diagnostics has received comparatively little explicit theoretical treatment. While truncation of Gaussian distributions is classical, the implications of mandatory truncation and order-statistic selection on innovation-based consistency diagnostics have not been explicitly characterized in the Kalman tracking literature. In particular, existing analyses typically focus on detection performance or association accuracy, rather than on the induced bias in innovation statistics themselves. Recent work on adaptive and learning enhanced Kalman filtering further highlights the reliance on innovation statistics for online consistency assessment and tuning [ 13 , 14 , 15 ] . These methods benefit from a precise understanding of how structural elements of the tracking pipeline, such as gating and association, affect the observed innovation process. The present work complements the existing literature by providing an explicit statistical characterization of gate-conditioned innovation moments and by showing that NN association introduces an unavoidable order-statistic bias. Unlike prior adaptive or robust filtering approaches, the results here isolate selection-induced effects under ideal modeling assumptions, thereby clarifying fundamental limitations in the interpretation of innovation-based diagnostics. The main contributions of this paper are summarized as follows: • Validation gating is interpreted as a statistical conditioning operation, and exact expressions are derived for the first- and second-order moments of the gate-conditioned innovation under linear–Gaussian assumptions. • It is shown that ellipsoidal gating induces a deterministic contraction of the innovation covariance that depends only on the gate threshold and the measurement dimension. • NN association is characterized as an order-statistic selection mechanism, and it is proven that this selection introduces an additional, unavoidable contraction of innovation energy, leading to an impossibility result: nominal innovation statistics cannot be preserved under nontrivial gating and association. • Closed-form results and illustrative examples are provided for the two-dimensional case, quantifying the combined effects of gating and association and highlighting their practical significance. The remainder of the paper is organized as follows. Section II introduces the problem formulation and the innovation model. Section III analyzes validation gating as a statistical selection mechanism and derives gate-conditioned innovation moments. Section IV discusses the implications of these results for innovation-based consistency diagnostics. Section V analyzes the additional bias induced by NN association. Section VI presents a two-dimensional case study with closed-form expressions