Filling in the Blanks: Applying Data Imputation in incomplete Water Metering Data
Dimitrios Amaxilatis, Themistoklis Sarantakos, Ioannis Chatzigiannakis, Georgios Mylonas · 2025
Read Filling in the Blanks: Applying Data Imputation in incomplete Water Metering Data by Dimitrios Amaxilatis, Themistoklis Sarantakos, Ioannis Chatzigiannakis, Georgios Mylonas (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-08882.txt.
Introduction
Amaxilatis, Sarantakos, Chatzigiannakis, and Mylonas present a study from twenty twenty-five on filling gaps in smart water meter data. Meters in a real water grid deployment stream detailed usage data. Technical faults leave gaps that skew decisions on leaks and upkeep. The team compare several gap-filling methods on that real dataset. Methods range from neighbor matching to forest models and neural networks. Transformer and recurrent network options join the comparison. Their checks weigh how filling affects accuracy and reliability of the data.
Good filling markedly improves insight quality from usage data, they find. Better data support leak spotting and predictive upkeep scheduling. The authors do not crown one method for all gaps. They stress matching the method to the gap pattern at hand.
Catalog record
| CREATOR | Dimitrios Amaxilatis, Themistoklis Sarantakos, Ioannis Chatzigiannakis, Georgios Mylonas |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 29 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2506-08882
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Filling in the Blanks: Applying Data Imputation in incomplete Water Metering Data PubID: pubid: 979-8-3503-5229-0/24/$31.00 ©2024 IEEE Thanks: This work was partially supported by the European Union under the Italian National Recovery and Resilience Plan (NRRP) of NextGenerationEU, partnership on “Telecommunications of the Future” (PE00000001 - program RESTART - CUP E83C22004640001) in the WITS and SPRINT focused project. This work was partially supported by the project SERICS (PE00000014 - CUP D33C22001300002) under the NRRP MUR program funded by the EU-NGEU. This work is supported by the European Union’s Horizon Program under the Agile and Cognitive Cloud edge Continuum management (AC3) project (Grant No. 101093129). Dimitrios Amaxilatis, Themistoklis Sarantakos, Ioannis Chatzigiannakis, Georgios Mylonas Abstract In this work, we explore the application of recent data imputation techniques to enhance monitoring and management of water distribution networks using smart water meters, based on data derived from a real-world IoT water grid monitoring deployment. Despite the detailed data produced by such meters, data gaps due to technical issues can significantly impact operational decisions and efficiency. Our results, by comparing various imputation methods, such as k-Nearest Neighbors, MissForest, Transformers, and Recurrent Neural Networks, indicate that effective data imputation can substantially enhance the quality of the insights derived from water consumption data as we study their effect on accuracy and reliability of water metering data to provide solutions in applications like leak detection and predictive maintenance scheduling. I Introduction In the era of smart cities and advanced utility management, the monitoring of water grids has become increasingly pivotal to ensuring efficient distribution, sustainability, and infrastructure reliability. Smart water meters, integral components of this intelligent monitoring framework, offer a granular view of water usage, potential leaks, and system performance. However, despite their sophistication, the occurrence of missing data due to various factors—ranging from technical malfunctions to data transmission errors— remains an open challenge that undermines the integrity and actionable insights that can be derived from the datasets produced by such infrastructure. Moreover, the significance of addressing missing data extends beyond mere data completeness. In the context of water grid monitoring, it impacts decision-making processes related to water management, leak detection, and predictive maintenance, all of which have profound implications for operational efficiency and environmental sustainability. Herein lies the critical role of data imputation methodologies designed to intelligently fill in missing data points and enhance the quality and usability of data collected by smart water meters. This work explores the application of advanced data imputation techniques within the domain of water grid monitoring. By integrating these methods with data from smart water meters, we aim to not only address the gaps in data but also augment the accuracy and reliability of the insights gained from such data. Through a comprehensive review of existing imputation methods and their application to water grid consumption data, we endeavor to provide a robust framework for utility managers, policymakers, and researchers to optimize water resource management and infrastructure planning. The data we use as a basis for our analysis come from water meters deployed in over 20 public buildings in Greece, collected over a period of more than 3 years between 2019 and 2024. Our methodology is structured into 3 distinct phases; we begin with data preprocessing, standardizing the collected data into a uniform format. During this stage, we identify missing values and generate supplementary information that aids in estimating such gaps. Furthermore, we partition the preprocessed data into training and validation datasets, which are crucial for subsequent analysis. The second phase involves training the models using the configured training dataset. We conduct extensive evaluations across multiple models and configurations to identify the most effective approach. In the last phase, we deploy these models to estimate data. Subsequently, we assess the estimations’ accuracy and reliability to ensure the outputs’ quality. This approach ensures a thorough analysis and validation of the data imputation process. In the following sections, we delve into the background of water grid monitoring, the challenges posed by missing data, and the pivotal role of smart water meters in contemporary water management. Subsequently, we will outline the methodology employed in applying data imputation techniques to water grid monitoring, followed by a detailed analysis of the results and their implications for the field. Through this investigation, we aim to contribute a meaningful discourse to the nexus of smart utilities and data science, providing actionable insights and methodologies to enhance the efficacy of water grid monitoring systems. II Related Work The use of quality data in water metering systems is of high importance when building applications that provide solutions such as leakage detection or consumption profiling/prediction. [ 1 ] presents a comprehensive Internet of Things (IoT)-based solution for real-time water management and leakage detection in water distribution networks (WDN). [ 2 ] explores an advanced IoT-based system for monitoring water leaks in irrigation systems using real-time data from a wireless sensor network integrating machine learning algorithms to process the collected data. [ 3 ] and [ 4 ] present novel methods for enhancing the efficiency of smart water distribution networks using edge computing within a LoRaWAN architecture. It introduces combining an IoT network simulator and a WDN simulator data, as well as a classification platform, which operates at the network’s edge to detect and manage water leakages efficiently. Similarly, [ 5 ] explores the use of IoT and machine learning to enhance the monitoring and management of water consumption in residential settings. The approach incorporates various sensors within a household to collect real-time data on water usage, which is then analyzed using machine learning techniques to identify and predict consumption patterns. In all the above cases, the quality of the collected data, and the completeness of the dataset used plays a crucial role in generating high-value predictions and estimations. The ability to have such a dataset is not always easy and guaranteed, due to the hassles of deploying and managing IoT systems and installations. As a result, the ability to fill in the gaps of the missing data is of paramount importance. The collection of studies on data imputation techniques encompasses a diverse array of innovative approaches tailored to diverse data scenarios. [ 6 ] provides an insightful review of imputation methods ranging from simple deletion to advanced machine learning (ML) techniques, focusing specifically on water distribution systems (WDS). It introduces a strategic “top-down bottom-up” method for selecting the most suitable imputation technique based on detailed data analysis and the characteristics of missing data, along with a comprehensive evaluation of each method’s strengths and weaknesses. [ 7 ] proposes enhancing the Multivariate Imputation by Chained Equations (MICE) method by reshaping sensor data to strengthen the correlation between observed and missing data, thereby improving imputation accuracy. This approach is demonstrated through its application to water quality monitoring data from Vermont, showing a significant improvement in model accuracy with at least a 23% increase in R² values. [ 8 ] examines the effectiveness of the K-Nearest Neighbors (K-NN) method across various scenarios with different missing data mechanisms. The study validates K-NN’s robustness in maintaining high data integrity and closely matching the accuracy of complete datasets, underscoring its utility in diverse conditions. [ 9 ] introduces MissForest, a non-parametric method that leverages the random forest approach