Explainability-Driven Feature Engineering for Mid-Term Electricity Load Forecasting in ERCOT’s SCENT Region

Abhiram Bhupatiraju, Sung Bum Ahn · 2025

Read Explainability-Driven Feature Engineering for Mid-Term Electricity Load Forecasting in ERCOT’s SCENT Region by Abhiram Bhupatiraju, Sung Bum Ahn (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-22220.txt.

Introduction

Abhiram Bhupatiraju and Sung Bum Ahn forecast power demand in a Texas grid region in a 2025 paper. Good mid term forecasts, up to a year ahead, guide upkeep plans, staffing, budgets, and market choices.

Four learning models were compared: a linear model, two tree based boosters, and a memory based neural network. Demand is shaped by weather and by time patterns in non linear ways, which the models must capture. A method called SHAP was used to show how much each input drove each forecast. Those readings then guided the choice and building of features, which lifted both clarity and accuracy of the forecasts.

Catalog record

CREATORAbhiram Bhupatiraju, Sung Bum Ahn
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE42 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2507-22220

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Explainability-Driven Feature Engineering for Mid-Term Electricity Load Forecasting in ERCOT’s SCENT Region Abhiram Bhupatiraju, Sung Bum Ahn Abstract Accurate load forecasting is essential to the operation of modern electric power systems. Given the sensitivity of electricity demand to weather variability and temporal dynamics, capturing non-linear patterns is essential for long-term planning. This paper presents a comparative analysis of machine learning models, Linear Regression, XGBoost, LightGBM, and Long Short-Term Memory (LSTM), for forecasting system-wide electricity load up to one year in advance. Midterm forecasting has shown to be crucial for maintenance scheduling, resource allocation, financial forecasting, and market participation. The paper places a focus on the use of a method called ”Shapley Additive Explanations” (SHAP) to improve model explainability. SHAP enables the quantification of feature contributions, guiding informed feature engineering and improving both model transparency and forecasting accuracy. Keywords and Abbreviations ML Machine Learning AI Artificial Intelligence ELF Electric Load Forecasting XAI Explainable Artificial Intelligence SHAP SHapley Additive Explanations LSTM Long Short-Term Memory CNN Convolutional Neural Network XGB eXtreme Gradient Boosting LR Linear Regression LightGBM Light Gradient Boosting Machine PLF Probabilistic Load Forecasting RNN Recurrent Neural Network FCM Fuzzy Cognitive Maps AR Autoregressive Model SVM Support Vector Machine MAE Mean Absolute Error MAPE Mean Absolute Percentage Error RMSE Root Mean Square Error ERCOT Electric Reliability Council of Texas SCENT South Central Texas Region 1 Introduction Load forecasting is a critical component for modern grid operation and planning. As global energy systems transition toward increased integration of renewable energy sources, face increasing demand, and even the proliferation of large data centers, grid operators must anticipate fluctuating demand with increased precision. Inaccurate forecasts can lead to inefficient scheduling, higher operational costs and even grid instability during peak load conditions. With snow storms and heatwaves becoming even more unpredictable as time goes on, the need for robust forecasting models increases in the context of smart grids, where system flexibility and ability to grasp non-linear patterns grows paramount. Mid-term load forecasting (MTLF), which generally refers to forecast horizons ranging from one week to one year, plays a unique role within the broader forecast spectrum. Although short-term forecasting is essential for real-time grid operations and long-term forecasting supports infrastructure development, MTLF is indispensable for maintenance scheduling, financial planning, energy generation, and regulatory compliance Yin et al. 2023 . Unlike Short-Term Forecasting it remains more volatile due to seasonal, behavioral, and climatic uncertainty. Traditional statistical models, such as ARIMA and exponential smoothing, often struggle with the non-linear and multi-scale patterns inherent to MTLF, creating space for the application of machine learning approaches. Ghiassi et al. 2006 ; Yin et al. 2023 . Recent advances in machine learning (ML) and deep learning (DL) have significantly improved prediction in MTLF applications. These models are well suited to capture nonlinear relationships, temporal dependencies, and high-dimensional feature interactions that traditional approaches fail to model effectively Yin et al. 2023 ; Waheed et al. 2024 . Techniques such as linear regression and ARIMA cannot be obtained. On the other hand, gradient-boosted trees (e.g. XGBoost and LightGBM) and recurrent neural networks (e.g. LSTM) have demonstrated superior predictive accuracy across various time horizons and geographies Waheed et al. 2024 ; Ghiassi et al. 2006 . Despite their empirical success, the practical deployment of these models is often hindered by their ”black-box” nature, which limits operator trust, reduces transparency, and impairs the ability to effectively feature engineer. Adding models with greater complexity was found to be inversely proportional to ideas such as explainability Baur et al. 2024 . To address these concerns, the field has seen growing interest in explainable artificial intelligence (XAI) techniques that aim to make ML/DL models more transparent. Among them, SHAP (SHapley Additive exPlanations) has emerged as a leading framework for interpreting model outputs, offering global and local explanations of feature contributions Baur et al. 2024 ; Neubauer et al. 2025 . Recent research highlights the importance of integrating interpretability into load forecast pipelines to support human decision making and improve model diagnostics Neubauer et al. 2025 ; Baur et al. 2024 . However, many existing studies stop at model interpretation and do not take advantage of explainability to improve forecast accuracy, particularly during peak load periods, where precise predictions are most crucial. Explainability is one of the most critical properties for machine learning models used in high-stakes domains, such as power systems. As defined by Baur et al. 2024 , explainability pertains to the extent to which the internal mechanics of a model can be understood and communicated. Increasing explainability is necessary to improve model performance and understanding feature importance. Despite the growing use of ML in load forecasting, many studies treat these models as black boxes, limiting their adoption in the real world Baur et al. 2024 . Our work builds on this foundation by adopting SHAP, a model-agnostic explainability method rooted in cooperative game theory. Originally developed to fairly attribute payouts to players in a game, SHAP has been adapted to machine learning to quantify the contribution of each feature to a prediction using Shapley values. In the context of electricity load forecasting, SHAP enables not only post hoc explainability but also deeper diagnostic insights into model behavior. It is able to do various calculations to provide a numerical value that each variable or feature has contributed to the prediction. In this study, we leverage SHAP to identify the most influential characteristics during extreme peak demand events, traditionally the most difficult forecast scenarios. By using these insights we are able to understand which features work positively to predict peak demands and which hurt our predictions. We are also able to engineer new features and refine the modeling pipeline. Additionally, our study aims to show that SHAP can be used not just for transparency, but also as a tool to improve mid-term forecast accuracy where it matters most. Midterm load forecasting has evolved through complementary advances in modeling techniques, explainability tools, and feature engineering. Our work builds upon these efforts by using SHAP explainability not only for interpretation but also as a tool for targeted feature engineering—particularly for improving peak load forecasts, which remain underexplored in existing literature. 1.1 Foundational MTLF Models: From Deep Learning to Hybrid Architectures Oreshkin et al. introduced the N-BEATS architecture for time series forecasting, replacing classical seasonality modeling with deep residual blocks that learn patterns directly from data Oreshkin et al. 2020 . Its successor, N-BEATS* Kasprzyk et al. 2024 , improved accuracy through a novel loss function and post-processing layer. While both enhance midterm accuracy over large regions, they remain uni variate and cannot incorporate exogenous drivers such as temperature or holiday effects—an essential limitation for electricity demand modeling. Zhang et al. proposed a hybrid framework combining seasonal decomposition with LSTM and XGBoost to capture linear trends and nonlinear residuals Kasprzyk et al. 2024 . Similarly, Pełka and Dudek introduced pattern-based LSTMs that encode structural seasonality into symbolic forms to stabilize long-horizon forecasts Pełka and Dudek 2020 . These models marked a shift toward hybrid architectures but lacked explainability or diagnostic tools to guide feature design. Zimmermann and Ziel proposed a Generalized Additive Model (GAM) incorporating holiday-aware splines and autoregressive terms for interpretable midterm forecasting across Europe Zimmermann and Ziel 2024 . While

Read the complete text