Neural Network State-Space Estimators

Minxing Sun, Li Miao, Qingyu Shen, Yao Mao, and Qiliang Bao · 2025

Read Neural Network State-Space Estimators by Minxing Sun, Li Miao, Qingyu Shen, Yao Mao, and Qiliang Bao (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-25959.txt.

Introduction

Minxing Sun and four co-authors rebuild state estimation so the neural network itself becomes the thing being estimated, in a 2025 paper. Classical estimators track a moving target through a hand-built model of its motion, which is laborious and brittle when dynamics change. Pure neural estimators avoid the model but demand huge precomputed training sets and ignore estimation theory. The authors propose a unified structure with no target model at all.

Inside it, the network's inputs and all its weights count as hidden states, estimated online as data arrives. Three classic estimators, extended Kalman, unscented Kalman, and particle, were each installed in the structure. Benchmarks against seven leading neural estimators across three scenarios test whether the marriage of theory and learning keeps the best of both.

Catalog record

CREATORMinxing Sun, Li Miao, Qingyu Shen, Yao Mao, and Qiliang Bao
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE34 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2509-25959

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Neural Network State-Space Estimators Minxing Sun, Li Miao, Qingyu Shen, Yao Mao, and Qiliang Bao Abstract Classical state estimation algorithms rely on predefined target’s state-space model, which complicates model derivation and limits adaptability when system dynamics change. Neural network based estimators offer a data-driven alternative, but rarely fuse classical estimation theory into their structure and demand large, pre-computed training sets. To overcome these limitations, we propose a unified state-space structure without target’s state-space model and treats both the input-layer activations and all network weights as latent states to be estimated online. We instantiate this nonlinear model with three canonical estimators—the extended Kalman estimator, the unscented Kalman estimator, and the particle estimator—to simulate different neural network and demonstrate its generality. We then benchmark our approach against seven leading neural network estimators across three representative scenarios. Results show that our neural network state-space estimators not only retain the robust learning capability, but also match or exceed the accuracy of both classical and pre-trained neural network methods. (Code, data, and more result: github.com/ShineMinxing/PaperNNSSE.git ) I Introduction State estimation algorithms are extensively utilized across various engineering disciplines [ 1 , 2 , 3 ] . For example, in electro-optical tracking systems [ 4 , 5 ] , they are essential for mitigating optical path disturbances caused by atmospheric turbulence [ 6 ] , compensating for delays in hardware and image processing pipelines [ 7 ] , and filtering out signal noise generated by sensors and target detection mechanisms [ 8 ] . The fundamental principle of these algorithms comprises three stages: predicting the next state using the current estimate and target’s state-space model, predicting the corresponding observation via the observation equation, and correcting the state estimate based on the deviations between the actual observation and its prediction. Estimators can be categorized based on how they establish the relationship between observation deviations and state deviations. One category comprises methods such as the Extended Kalman Estimator (EKE), which computes partial derivatives from the state-space model. The primary limitation of EKE is the complexity involved in deriving analytical solutions for these partial derivatives, particularly in highly nonlinear models [ 9 , 10 , 11 , 12 , 13 ] . Another category includes the Unscented Kalman Estimator (UKE), which approximates the state distribution by assigning predefined deviations to each estimated state and evaluating the resulting variations in observations. However, UKE faces challenges in accurately capturing the relationships between coupled estimated states and observations [ 14 , 15 , 16 , 17 , 18 , 19 , 20 ] . The third category is represented by Particle Estimator (PE), which introduces random perturbations to the estimated states and assesses the resulting observation variations. While PE is better suited for handling nonlinearities, its main drawback is the high computational cost associated with the large number of particles required, which can impede real-time performance [ 21 , 22 , 23 , 24 ] . All three estimation methods rely on predefined state-space models, and even the robust estimators designed to mitigate model inaccuracies can only compensate for a limited range of errors [ 25 , 26 , 7 ] . In practical applications, defining an accurate state-space model for the target system is often inherently challenging, particularly when tracking non-cooperative targets. The rapid advancement and widespread adoption of artificial neural network methodologies have profoundly impacted time-domain signal processing, enabling significant breakthroughs in estimation and prediction tasks [ 27 , 28 , 29 , 30 , 31 , 32 ] . Recurrent Neural Network (RNN), introduced by Hopfield [ 33 ] , were enhanced by Rumelhart, Hinton, and Williams [ 34 ] with the backpropagation algorithm, facilitating more efficient training. Subsequent improvements by Jordan and Elman [ 35 , 36 ] incorporated feedback loops, enhancing the ability to model temporal dependencies [ 37 , 38 , 39 ] . However, RNNs struggled with vanishing and exploding gradients, challenges addressed by Hochreiter [ 40 ] with Long Short-Term Memory (LSTM) networks, and further refined by Schmidhuber [ 41 ] through the addition of forget gates [ 42 , 43 ] . Gated Recurrent Unit (GRU), proposed by Kyunghyun [ 44 ] and validated by Junyoung [ 45 ] , offered a simplified yet effective alternative to LSTM [ 46 , 47 , 48 ] . Convolutional Neural Network (CNN), initially developed by LeCun [ 49 ] for handwritten digit recognition, have been extended to Recurrent Convolutional Neural Networks [ 50 ] to capture spatial and temporal dependencies, significantly improving prediction accuracy in applications such as vehicle and human trajectory forecasting [ 51 , 52 , 53 , 54 ] . Temporal Convolutional Network (TCN), introduced by Colin [ 55 ] , utilize pooling and upsampling to capture long-term motion patterns, enhancing capabilities in action segmentation and detection [ 56 , 57 , 58 , 59 ] (Chen, 2020; Kaiyu, 2022). Neural Ordinary Differential Equation (NeuralODE), proposed by Ricky [ 60 ] , represent neural network layers through differential equations, facilitating deeper and more efficient networks for tasks like robotic path planning and travel time estimation [ 61 , 62 , 63 ] . Transformer architectures, introduced by Ashish [ 64 ] , leverage self-attention mechanisms to model long-term dependencies while reducing computational complexity. Their applications span maneuvering target tracking, autonomous vehicle trajectory prediction, and 3D human pose estimation [ 65 , 66 , 67 , 68 , 69 ] . Traditional estimation algorithms have also been integrated into neural networks to optimize training processes. Early efforts by Singhal, Puskorius, and Pérez-Ortiz [ 70 , 71 , 72 , 73 ] applied EKE to train multilayer perceptrons and RNN, enhancing training efficiency. More recent integrations of UKE and PE have improved the training of feedforward and recurrent networks in applications such as voice classification and evapotranspiration estimation [ 74 , 75 ] . However, in these studies, estimation algorithms primarily treated the backpropagation algorithm as state-space equations, serving an auxiliary role rather than being the primary mechanism for state estimation. Consequently, the effectiveness of estimation algorithms remains limited and warrants further enhancement [ 76 ] . To address these limitations, our proposed algorithm introduces a novel state-space model that treats input layer nodes and all weight parameters as estimated states, effectively simulating a neural network within a state estimation framework. The main contributions of this work are as follows: • Neural Network State-Space Model (NNSSM): We propose a NNSSM that simulates a neural network, enabling real-time learning and pretraining. The estimation algorithm works wothout target’s state-space models and can be tailored by adjusting the input layer length and prediction steps to meet engineering requirements. • Neural Network State-Space Estimator (NNSSE): The versatility of the proposed NNSSM is validated through the implementation of three representative estimation algorithms—EKF, UKE, and PF—demonstrating its broad applicability. • Flexible Architecture Configuration: The NNSSM can accommodate arbitrary neural network structures—adjusting the number of layers, the number of neurons per layer, connectivity patterns, and activation functions—to meet diverse application requirements. • Comparative Performance Evaluation: We compare the estimation performance of our NNSSEs with various neural network architectures, including RNN, LSTM, GRU, TCN, NeuralODE, and Transformer. • Comprehensive Testing: Extensive tests were conducted on generated sine trajectories, a laboratory dual-reflection mirror

Read the complete text