State Estimation and Control of Dynamic Systems from High-Dimensional Image Data
Ashik E Rasul, Hyung-Jin Yoon · 2025
Read State Estimation and Control of Dynamic Systems from High-Dimensional Image Data by Ashik E Rasul, Hyung-Jin Yoon (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-05375.txt.
Introduction
Ashik E Rasul and Hyung-Jin Yoon address, in a 2025 paper, the problem of controlling a dynamic system when its true state cannot be measured directly. Accurate state estimates are needed for good control, but direct measurement is often impractical. Their approach learns states from sequences of images together with the actions taken.
The architecture combines convolutional neural networks, which extract spatial features from each image, with gated recurrent units, which model how the system changes over time. The learned state representations are then used to train a reinforcement learning agent based on a Deep Q-Network. Experiments show that the agent can estimate state and control the system in real time without access to ground-truth states. The authors also provide a quantitative method for judging how accurate the learned states are and how that accuracy affects control performance.
Catalog record
| CREATOR | Ashik E Rasul, Hyung-Jin Yoon |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 21 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2506-05375
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
State Estimation and Control of Dynamic Systems from High-Dimensional Image Data Ashik E Rasul, Hyung-Jin Yoon Abstract Accurate state estimation is critical for optimal policy design in dynamic systems. However, obtaining true system states is often impractical or infeasible, complicating the policy learning process. This paper introduces a novel neural architecture that integrates spatial feature extraction using convolutional neural networks (CNNs) and temporal modeling through gated recurrent units (GRUs), enabling effective state representation from sequences of images and corresponding actions. These learned state representations are used to train a reinforcement learning agent with a Deep Q-Network (DQN). Experimental results demonstrate that our proposed approach enables real-time, accurate estimation and control without direct access to ground-truth states. Additionally, we provide a quantitative evaluation methodology for assessing the accuracy of the learned states, highlighting their impact on policy performance and control stability. Figure 1: Proposed framework for dynamic system state prediction and RL training Introduction Autonomous systems operating in real-world environments, such as unmanned aerial vehicles (UAVs) and autonomous vehicles, frequently rely on actionable state information extracted from high-dimensional sensory data like images or videos. Humans possess a natural perceptual capability to interpret sequential visual cues and infer temporal properties such as velocity. However, replicating this ability in artificial systems involves two key challenges: extracting important spatiotemporal features from high-dimensional data and estimating the true state of the system from noisy observations. Deep Neural Networks (DNNs) have consistently demonstrated superior performance over traditional computer vision methods ( Grigorescu et al. 2020 ) in perception-based tasks for autonomous systems. Convolutional neural networks (CNNs) are widely used for extracting hierarchical spatial features from images ( LeCun, Bengio, and Hinton 2015 ) . To capture sequential dependencies in visual data, recurrent neural networks (RNNs), especially gated recurrent units (GRUs), are widely adopted due to their effectiveness in modeling temporal relationships with relatively lightweight architectures ( Cho et al. 2014 ) . Leveraging the universal function approximation capability of fully connected neural networks (FCNNs) ( Hornik 1991 ) , the latent representations produced by GRUs can be mapped to interpretable, low-dimensional state predictions of dynamic systems, bridging the gap between high-level perception and actionable insights required for optimal control. Conventional deep reinforcement learning (RL) algorithms, such as Deep Q-Networks (DQN), often assume full access to the environment’s true state vector. While this assumption simplifies policy learning, it does not reflect real-world conditions, where direct measurement of all states is typically expensive or challenging. In such systems, the true state values must instead be estimated through processing one or multiple sensor inputs. This research addresses these challenges by proposing a perception-based controller design framework (as shown in Figure 1 ) that integrates spatiotemporal feature extraction with learned state estimation. Our framework employs a CNN to extract spatial features from individual image frames. These features, combined with corresponding actions, are processed through a GRU to capture temporal dependencies. An FCNN subsequently maps these learned representations to physically meaningful, low-dimensional state estimates. By processing sequences of frames and incorporating action history, the model implicitly captures latent dynamics like velocity, filtering out irrelevant information and emphasizing task-relevant features. The estimated states serve as inputs to a reinforcement learning (RL) agent to train an optimal policy. To the best of our knowledge, this framework represents the first approach explicitly designed to extract interpretable state information for optimal controller design from raw visual inputs. Our key contributions are summarized as follows: • We develop a perception-based reinforcement learning framework 1 1 1 https://github.com/arasul42/cartpoleStatePred that extracts interpretable state predictions from high-dimensional image data. • We demonstrate that an RL agent trained on these predicted states achieves comparable performance to an RL agent trained using full state information. • We present a quantitative evaluation methodology to independently assess state prediction accuracy and controller performance. Background Perception-based control relies heavily on compressing high-dimensional observations into low-dimensional representations that preserve task-critical temporal features. To process grid-structured data such as images, Convolutional Neural Networks (CNNs) are most widely used. They utilize convolutional layers with learnable filters to extract local and hierarchical features from raw input data, followed by non-linear activation functions like ReLU to introduce complexity into the model. CNNs with backpropagation were successfully applied to identify handwritten ZIP codes ( LeCun et al. 1998 ) . However, when the number of layers in the neural network is increased, traditional backpropagation becomes challenging. They face problems such as local optima, gradient vanishing, gradient exploding or overfitting. The introduction of ImageNet and multi-hidden-layer pretraining ( Krizhevsky, Sutskever, and Hinton 2012 ) shows the capability of CNNs in large scale image classification tasks. However, despite their effectiveness in capturing spatial patterns, standard CNNs are limited in modeling long-range temporal dependencies in sequential data, as they lack an explicit mechanism to capture the order or temporal dynamics inherent in time-series. This shortcoming motivates the use of recurrent architectures for tasks where temporal context plays a critical role. Traditional Fully Connected Neural Networks (FCNNs) demonstrated high accuracy in regression tasks ( Chemali et al. 2018 ) but suffer from an explosion in the number of parameters when processing long sequences. Recurrent Neural Networks (RNNs) inherently handle sequential data better and have demonstrated superior performance compared to FCNNs. However, they face gradient vanishing and exploding issues for long input sequences ( Bengio, Simard, and Frasconi 1994 ) . To overcome these limitations, Gated RNNs, such as Long Short-Term Memory (LSTM) and Gated Recurrent Units (GRU), introduce gating mechanisms to regulate information flow and capture long-term dependencies. LSTMs have shown exceptional performance in various domains, including time series forecasting ( Siami-Namini, Tavakoli, and Namin 2019 ) , machine translation ( Sutskever, Vinyals, and Le 2014 ) , and healthcare diagnostics ( Balaji et al. 2021 ) . However, their complex architecture and high computational cost hinder real-time deployment. To balance computational efficiency and accuracy, GRUs were developed, offering comparable performance with a simpler structure ( Yang et al. 2019 ) , making them more suitable for real-time applications. Reinforcement learning (RL) is a branch of machine learning in which agents learn to make decisions by interacting with an environment to maximize cumulative rewards ( Sutton, Barto et al. 1998 ) . Q-learning is a widely used RL algorithm that enables agents to learn a Q-function, which estimates the expected return of taking a particular action in a given state. The introduction of the Deep Q-Network (DQN) ( Mnih et al. 2015 ) combined Q-learning with deep neural networks, allowing agents to learn control policies directly from high-dimensional sensory inputs such as raw pixel images. DQN incorporates key innovations such as experience replay—where the agent’s experiences are stored and randomly sampled during training—and a target network, which is updated periodically to stabilize learning and prevent divergence. Conventional implementations of Deep Q-Networks (DQN) typically assume access to full state information ( Raffin et al. 2021 ) . However, in real-world scenarios, such complete observability is often unattainable due to sensor limitations or occlusions. To address this, reinforcement learning based on low-dimensional latent representations has gained significant traction ( Lee et al. 2020 ) . Despite their success, the lack of interpretability in these latent-based approaches poses a challenge for deployment in safety-critical or high-stakes applications.