Robust Contextual State Estimation with Limited Measurement Data

J. G. De la Varga, J. M. Morales, S. Pineda · 2025

Read Robust Contextual State Estimation with Limited Measurement Data by J. G. De la Varga, J. M. Morales, S. Pineda (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-08767.txt.

Introduction

De la Varga, Morales, and Pineda tackle in 2025 state estimation for power distribution grids with scarce live data. Live measures are often too sparse to pin the grid state, while past data go stale as conditions shift. Both gaps hit at once in daily grid work.

A neighbor-based estimator first guesses the spread of delayed measures from current context. A robust version then guards against errors in that guess by reweighting neighbors under set rules. The guard problem rewrites as one level with like cost to the base tool. A data-driven rule picks the guard strength online, and grid tests check the gains.

Catalog record

CREATORJ. G. De la Varga, J. M. Morales, S. Pineda
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE48 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2507-08767

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Robust Contextual State Estimation with Limited Measurement Data J. G. De la Varga, J. M. Morales, S. Pineda Abstract The increasing integration of distributed energy resources (DERs) is transforming power systems into complex, decentralized networks, particularly at the distribution level, where active distribution networks (ADNs) introduce new challenges for monitoring and control. Accurate state estimation in these systems is complicated by limited and heterogeneous measurement infrastructures, where real-time data from supervisory control and data acquisition (SCADA) and micro-phasor measurement units ( μ \mu -PMUs) is sparse, and slower data from Advanced Metering Infrastructure (AMI) is abundant but delayed. In this context, historical data becomes a key asset, yet its effective use requires robust methods that account for its relevance and variability. This paper proposes a novel data-driven methodology for state estimation under limited real-time observability. We first introduce a non-parametric estimator that leverages nearest neighbors to approximate the conditional expectation of delayed measurement residuals. Building on this, we develop a robustified extension that accounts for ambiguity in the estimated conditional distribution, mitigating the impact of sample sparsity and uncertainty. Numerical experiments demonstrate that the proposed approach brings substantial benefits in accuracy and robustness, particularly in scenarios with scarce and noisy measurements. I Introduction Power systems are undergoing a profound transformation, shifting from traditionally centralized architectures toward increasingly complex and decentralized networks. This evolution is driven by the growing penetration of DERs, such as rooftop solar, energy storage, and electric vehicles, which introduce significant operational uncertainty due to their stochastic behavior both in the generation and in the consumption. At the distribution level in particular, ADNs characterized by bidirectional power flows, tighter interconnections and changing topology, emerge as a new paradigm, making their operation and monitoring substantially more challenging than in the past [ 1 ] . Accurate state estimation in such uncertain environments remains a critical task for real-time monitoring and control. However, measurement infrastructure in distribution systems is typically sparse and heterogeneous [ 2 ] : fast but sparse measurements like SCADA systems and \mu -PMUs are received in real-time but exist in limited number, while slow but abundant measurements consist of more widely available data from AMI such as Smart Meters (SM). This limitation in granularity and latency leads most existing approaches to favor static estimation frameworks over dynamic and forecasting-aided ones [ 3 ] . In the face of limited real-time measurements, historical data becomes a valuable resource for inferring the current state of the system. However, in modern distribution networks, even assuming that the grid topology and parameters are known, we typically face a double information shortage . First, the available real-time measurements are often insufficient to guarantee system observability, especially given the sparse and uneven deployment of sensing infrastructure. Second, the historical data itself tends to be limited in both relevance and size, due to the rapid evolution of distribution systems: network topologies are frequently reconfigured and new DERs, such as rooftop solar units, are continuously being installed. As a result, previously collected data may quickly become outdated or uninformative. These two features, namely, structural unobservability and limited historical data, characterize the operational setting considered in this work. While several approaches to distribution system state estimation have addressed information limitations, a comprehensive treatment of the combined scenario involving structural unobservability and limited historical data remains largely unexplored in the literature, despite its growing practical relevance. In this paper, we propose estimation methods that are explicitly designed to operate under such information-constrained conditions, offering improved robustness and accuracy where conventional methods are not implementable or tend to fail. Such a dramatic information shortage motivates the use of contextual learning approaches, which aim to exploit similarities between present and past measurements to approximate the conditional distribution of the underlying system state. However, this approximation is highly susceptible to inference errors, particularly when the available historical dataset is sparse or when the same real-time measurements may correspond to a broad range of possible system states [ 4 ] . A prevalent strategy in the literature to handle previous challenges is the adoption of Bayesian frameworks. For instance, [ 5 ] employs a deep neural network to learn mappings from limited real-time measurements to system states, combining it with a Bayesian state estimation algorithm to improve robustness against bad data. In [ 6 ] , Gaussian process-based load forecasts are integrated with a linearized three-phase power flow model, enabling voltage predictions that are subsequently updated via linear Bayesian estimation. The authors in [ 7 ] introduce a scalable and robust Gaussian process regression technique that supports accurate pseudo-measurement inference and state estimation using a limited number of measurements. Alternatively, some works adopt data-driven corrections to physical models. For example, [ 8 ] proposes a data-enhanced linear power flow formulation to improve the accuracy of state estimation functions, in which the elements of the admittance matrix are adjusted based on available measurements. Robustness is ensured through a Huber estimator, allowing convergence even under uncertain model parameters, while zero-injection nodes are utilized to enhance observability. In a different direction, [ 9 ] formulates the problem as a mixed-integer linear program (MILP), treating both system states and load consumptions as decision variables. The aim is to maximize state estimation accuracy under power flow constraints, with pseudo-measurements from historical data setting feasible bounds for load variables. With a similar mathematical approach, [ 10 ] addresses the challenge of topology identification in unobservable distribution systems with limited measurements. A compressive sensing framework that jointly estimates system state and network topology is proposed, formulated as a MILP with auxiliary variables to eliminate non-linearity and a convex relaxation to enable faster solution times. Finally, in [ 11 ] a modified weighted least squares (WLS) estimator is designed to operate effectively with very few real measurements and without relying on pseudo-measurements, even in typically unobservable scenarios. The method retains the advantages of a mature and widely used state estimation approach while enhancing observability by incorporating allocation factors, whose key assumption is that the power consumed (or generated) within each cluster of buses would be always at the same per-unit level (with respect to the rated one) for each node of the grid. Despite their contributions, the described literature either assumes access to large amounts of historical data [ 5 , 6 , 7 , 8 ] , relies on approximated models [ 8 , 9 , 10 ] , or introduces structural simplifications that reduce the complexity of the state estimation [ 11 ] . Against this background, the contribution of this paper is threefold: • We first introduce a non-parametric data-driven state estimator which leverages nearest neighbors to approximate the conditional expectation of the delayed measurements’ residuals given the available measurements. The estimator is expressed as a contextual stochastic optimization problem. • We then present a robustified extension of this state estimator, which optimizes against ambiguity on the conditional distribution within a vicinity of the one estimated by the nearest neighbors. This novel formulation aims to reduce sensitivity to sample uncertainty and sparsity, inherent in the presented scenario of limited real-time data. Through a set of numerical experiments, we analyze the conditions under which this added robustness translates into improved estimation performance. • Finally, as a

Read the complete text