Learning from Change: Predictive Models for Incident Prevention in a Regulated IT Environment Conference: 2026 IEEE/ACM 48th International Conference on Software Engineering; April 12–18, 2026; Rio de Janeiro, Brazil 2026 IEEE/ACM 48th International Conference on Software Engineering (ICSE-SEIP ’26), April 12–18, 2026, Rio de Janeiro, Brazil DOI: 10.1145/3786583.3786876 ISBN: 979-8-4007-2426-8/2026/04 CCS: Software and its engineering Software reliability CCS: Software and its engineering Software post-development issues CCS: Computing methodologies Natural language processing CCS: Computing methodologies Supervised learning by classification CCS: Computing methodologies Classification and regression trees CCS: Social and professional topics Governmental regulations

Eileen Kapel, Jan Lennartz, Luis Cruz, Diomidis Spinellis, Arie van Deursen · 2026

Read Learning from Change: Predictive Models for Incident Prevention in a Regulated IT Environment Conference: 2026 IEEE/ACM 48th International Conference on Software Engineering; April 12–18, 2026; Rio de Janeiro, Brazil 2026 IEEE/ACM 48th International Conference on Software Engineering (ICSE-SEIP ’26), April 12–18, 2026, Rio de Janeiro, Brazil DOI: 10.1145/3786583.3786876 ISBN: 979-8-4007-2426-8/2026/04 CCS: Software and its engineering Software reliability CCS: Software and its engineering Software post-development issues CCS: Computing methodologies Natural language processing CCS: Computing methodologies Supervised learning by classification CCS: Computing methodologies Classification and regression trees CCS: Social and professional topics Governmental regulations by Eileen Kapel, Jan Lennartz, Luis Cruz, Diomidis Spinellis, Arie van Deursen (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-13462.txt.

Introduction

Kapel and four colleagues build an incident-risk score for IT change at a large bank. Many IT incidents stem from changes, so the team predicts which changes risk incidents before deployment, aiding planning. To meet rules for auditability and explainability, they use SHAP values for feature insight. On a one-year real-world dataset, LightGBM beats the rule-based process and two rival models, especially with team metrics capturing organisational context.

Catalog record

CREATOREileen Kapel, Jan Lennartz, Luis Cruz, Diomidis Spinellis, Arie van Deursen
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE61448
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.13462

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Learning from Change: Predictive Models for Incident Prevention in a Regulated IT Environment Conference: 2026 IEEE/ACM 48th International Conference on Software Engineering; April 12–18, 2026; Rio de Janeiro, Brazil 2026 IEEE/ACM 48th International Conference on Software Engineering (ICSE-SEIP ’26), April 12–18, 2026, Rio de Janeiro, Brazil DOI: 10.1145/3786583.3786876 ISBN: 979-8-4007-2426-8/2026/04 CCS: Software and its engineering Software reliability CCS: Software and its engineering Software post-development issues CCS: Computing methodologies Natural language processing CCS: Computing methodologies Supervised learning by classification CCS: Computing methodologies Classification and regression trees CCS: Social and professional topics Governmental regulations Eileen Kapel, Jan Lennartz, Luis Cruz, Diomidis Spinellis, Arie van Deursen Abstract . Effective IT change management is important for businesses that depend on software and services, particularly in highly regulated sectors such as finance, where operational reliability, auditability, and explainability are essential. A significant portion of IT incidents are caused by changes, making it important to identify high-risk changes before deployment. This study presents a predictive incident risk scoring approach at a large international bank. The approach supports engineers during the assessment and planning phases of change deployments by predicting the potential of inducing incidents. To satisfy regulatory constraints, we built the model with auditability and explainability in mind, applying SHAP values to provide feature-level insights and ensure decisions are traceable and transparent. Using a one-year real-world dataset, we compare the existing rule-based process with three machine learning models: HGBC, LightGBM, and XGBoost. LightGBM achieved the best performance, particularly when enriched with aggregated team metrics that capture organisational context. Our results show that data-driven, interpretable models can outperform rule-based approaches while meeting compliance needs, enabling proactive risk mitigation and more reliable IT operations. † † cc-license: by 1. Introduction At present, major industries such as finance, healthcare, and retail are increasingly reliant on software and related services. However, the reliability and availability of these services are often compromised by incidents : unplanned interruptions to a service or reductions in service quality ( 18 ) . A significant portion of incidents stem from IT changes, which involve additions, modifications, or deletions to existing IT applications ( 18 ) . As reported by Google ( 6 ) , roughly 70% of outages in its live systems are due to such changes. To monitor these change-induced incidents, we often refer to the change failure rate, which measures the percentage of deployments causing a failure in production ( 13 ) . It is important to ensure service reliability and availability, because ineffective incident management can lead to customer dissatisfaction, financial losses, and reputational damage. For example, infrastructure failures may cost $100,000 per hour, while critical application failures can reach $1 million per hour ( 12 ) . To mitigate these risks and prevent severe business disruptions, financial institutions in Europe are required to follow rigorous change and incident management processes, ensuring that all changes are carefully assessed and auditable ( 35 ; 3 ) . Identifying potential incident-inducing changes before deployment can help prevent incidents and ensure service reliability ( 14 ) . Engineers prefer proactive measures ( 20 ) , such as enhancing testing and issue detection early in the development process, a practice known as “Shift Left”. This approach improves service reliability by detecting incident-causing bugs before they reach production ( 14 ) . Incident prevention entails actively identifying potential failures and forecasting severe outages by using historical data and analytics to predict and mitigate risks ( 36 ) . Predictive incident management is enabled by artificial intelligence for IT Operations (AIOps), which uses machine learning (ML) and big data mining to forecast potential system malfunctions by analysing historical patterns ( 36 ) . Ensuring trust, involving humans in the decision-making process, and providing interpretability and explainability of AIOps solutions is essential for gaining confidence from industry practitioners ( 36 ) . Interpretable models are often favoured over black-box models, even when slightly less accurate, as they offer great transparency and support informed decision-making. Deploying predictive incident management in a highly regulated financial environment introduces unique challenges. Change and incident management processes in financial institutions are tightly governed by regulatory standards that demand compliance, auditability, and traceability ( 35 ; 9 ; 3 ) . These constraints limit the use of black-box models like deep neural networks or large language models ( 38 ) , which often lack the transparency needed for both regulatory approval and practitioner trust. In this work, we address these challenges by developing an approach that predicts the probability that an IT change will cause incidents. We focus on boosted tree-based classifiers (HGBC, LightGBM, and XGBoost), which are proven in practice ( 31 ) , well-suited for tabular IT management data and support post-hoc interpretability through established methods such as SHapley Additive exPlanations (SHAP). Our approach generates an incident prediction score for each planned change, enabling engineers to assess deployment readiness. SHAP-based explanations provide feature-level transparency that supports user trust, informed decision-making and meets financial-sector audit requirements. Furthermore, we enrich the dataset with aggregated metrics that reflect team performance, such as change success rates, incident counts, and release outcomes, to evaluate their impact on model performance. A key design consideration is the trade-off between accuracy and explainability. Regulatory constraints make high-performing but opaque models unsuitable, as change-approval decisions must remain traceable and auditable. The resulting prediction scores produced by the models are intended as risk signals, helping teams prioritise which changes merit closer review. We evaluate our approach using a one-year dataset from our case company ING (International Netherlands Group), a multinational banking and financial services corporation. Building on insights from our prior work ( 21 ) , which analysed the characteristics of incident-inducing changes, we tailor our solution to the organisational and regulatory requirements of financial institutions. This ensures the model is not only accurate, but also usable, trustworthy, and compliant for engineers making deployment decisions – aspects that are essential in our context but have received little attention in previous work. Our main goal is to equip engineers with a predictive and explainable model that complies with regulatory standards, allowing them to identify high-risk changes during the assessment & planning phase and take preventive action to ensure reliable deployments. In particular, we seek to improve upon methods based on business rules and human effort, as currently employed at ING, by training ML models on historical data. We also investigate how team performance metrics can contribute to assessing incident prediction. We address the following research questions: • RQ1 : What is, in practice, the performance of a rule-based approach for obtaining the incident prediction score for deployment changes? • RQ2 : How can we use data-driven ML models for incident prediction scoring, and how does their performance compare to the rule-based approach? • RQ3 : What is the effect of including aggregated team metric data on the accuracy of ML-based models for incident prediction scoring? In our study, we demonstrate how data-driven ML methods can be employed within a tightly regulated change management process, providing more effective risk assessment than the rule-based approaches currently used in practice. Among the evaluated models, LightGBM delivers the best performance, particularly when enriched with aggregated team metrics. The model’s predictions remain explainable, with influential factors including team metadata,

Read the complete text