Unifying Runtime Monitoring Approaches for Safety-Critical Machine Learning: Application to Vision-Based Landing
Mathieu Dario ✉, Florent Chenevier, Kévin Delmas, Joris Guerin, Jérémie Guiochet · 2026
Read Unifying Runtime Monitoring Approaches for Safety-Critical Machine Learning: Application to Vision-Based Landing by Mathieu Dario ✉, Florent Chenevier, Kévin Delmas, Joris Guerin, Jérémie Guiochet (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-26411.txt.
Introduction
Dario and four colleagues sorted scattered runtime checks for safety-critical machine learning into three clear types. One type, operating-domain monitoring (ODD), checks that operating conditions match what was expected. A second, out-of-distribution monitoring (OOD), rejects inputs unlike the training data. A third, out-of-model-scope monitoring (OMS), spots odd behavior from a model's inner states or outputs. The team tried the scheme on runway detection during landing, a demanding aeronautical case. Shared safety-oriented metrics then let different checks be compared on common ground.
Catalog record
| CREATOR | Mathieu Dario ✉, Florent Chenevier, Kévin Delmas, Joris Guerin, Jérémie Guiochet |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 56999 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.26411
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Unifying Runtime Monitoring Approaches for Safety-Critical Machine Learning: Application to Vision-Based Landing Mathieu Dario ✉, Florent Chenevier, Kévin Delmas, Joris Guerin, Jérémie Guiochet Abstract Runtime monitoring is essential to ensure the safety of ML applications in safety-critical domains. However, current research is fragmented, with independent methods emerging from different communities. In this paper, we propose a unified framework categorising runtime monitoring approaches into three distinct types: Operational Design Domain (ODD) monitoring, which ensures compliance with expected operating conditions; Out-of-Distribution (OOD) monitoring, which rejects inputs that deviate from the training data; and Out-of-Model-Scope (OMS) monitoring, which detects anomalous model behaviour based its internal states or outputs. We demonstrate the benefits of this categorization with a dedicated experiment on an aeronautical safety-critical application: runway detection during landing. This framework facilitates design of monitoring activities, with complementary categories of monitors, and enables evaluation and comparison of different monitors using common, safety-oriented metrics. 1 Introduction Recent advances in Machine Learning (ML) technologies have enabled their deployment across a range of domains, including safety-critical applications such as autonomous driving, robotic surgery, and aviation, where data-driven techniques promise enhanced automation and decision support. In particular, deep neural networks (DNNs) underpin state-of-the-art perception methods for tasks like image classification and object detection, interpreting complex sensor signals to provide state estimates used in decision and control. However, despite impressive achievements, DNNs inherently lack robustness and reliability [ 1 ] , with performance degrading under unforeseen or adverse inputs [ 18 ] , limited explainability [ 33 ] , and often excessive confidence when mispredicting [ 35 ] . To address these challenges, safety standards for ML integration in critical systems are evolving, exemplified by initiatives from entities such as the European Aviation Safety Agency (EASA) [ 13 ] . Runtime safety monitors, derived from the dependability community, have emerged as promising solutions to improve the reliability of ML functions. They are fault-tolerance mechanisms, continuously monitoring the behaviour, inputs, and outputs of ML components. Through early anomaly or failure detection, these monitors can trigger mitigation at runtime, providing an additional safeguard beyond traditional offline verification. While various methods have been developed to monitor the safety of ML components, they often come from distinct research communities. For instance, the dependability community and the ML community both provided approaches that rely on different concepts within different scopes (global system operational domain vs. ML-specific detection of anomalies). This fragmentation results in partial coverage of the safety risks, each methodology focusing on different aspects of the problem. The limitations arising from this lack of unified perspective are further examined in Section 2 . In this paper, we clarify the definitions of the key monitoring notions used across the different research communities, establishing a common understanding and terminology. We introduce a unified, multi-level framework called SwMF, to organise monitoring approaches according to the risks they address in the ML input space. Finally, we illustrate our framework through an experimental study on an aviation safety-critical application, namely Vision-Based Landing (VBL) [ 5 ] . We particularly focus on the sub-task of detecting the runway in an image taken during landing, as depicted in Figure 1 . Figure 1 : Illustration of the Vision-Based Runway Detection (VBRD) task. The rest of the paper is organized as follows. Section 2 presents a review of current approaches related to ML safety monitoring. Section 3 introduces our proposed formalism (SwMF), by redefining data domains and data threats to derive a threat-based unified classification of monitoring activities. Section 4 details our experimental study, illustrating how our framework can be implemented on a concrete use case. Finally, Section 5 presents our conclusions and perspectives. 2 Related Work In this section, we present previous contributions in the domain for runtime safety monitoring of ML functions. We first give an overview of the sector of traditional safety monitoring before presenting its adaptation to the specific case of ML models. 2.1 From Traditional Safety Monitoring to ML Monitoring In the dependability community [ 3 ] , safety monitors (SMs) are fault-tolerance mechanisms, observing system inputs, states, and outputs as well as environment states to ensure continued safe operation. They verify compliance with predefined safety properties , established via rigorous system-level safety analyses and formalized by domain experts. They are positioned at critical decision points within the system architecture, ensuring that only safe states are propagated downstream. This approach is well-established and effective when system and environment states can be measured and hazards can be identified and formalized. The integration of ML models in perceptions tasks, such as DNNs that estimate system states from sensor inputs (images, videos), brings the need for ML-specific monitors. Indeed, applying SMs to ML models presents significant challenges. Traditional SMs can be used in a black-box manner to monitor systems [ 32 ] , including ML systems, by leveraging redundant sources of observation to infer system or environment states and verify corresponding safety rules. However, this becomes particularly difficult when dealing with complex ML-based tasks such as computer vision, where no straightforward alternative exists to DNNs. Additionally, developing interpretable safety properties at the ML level is also problematic. Expressing rules or assertions based on internal model features, such as neuron values or layer activations, remains opaque and uninformative from a system safety perspective. To address these limitations, data-driven monitors have emerged. They learn statistical patterns observed during model training to correlate inputs, outputs, or internal features with possible failure risks. Despite recent advances, substantial unresolved questions remain, as pinpointed by Ferreira et al. [ 16 ] , such as identifying relevant threats to address, translating system safety properties to ML-level criteria, designing efficient mechanisms for both detecting and mitigating ML failures, and finally evaluating monitoring impact on system safety. Diverse communities have addressed the question of ML safety monitoring, in rather independent ways. System-level approaches, like SMs, were developed by the dependability community, for instance. ML-level approaches were developed in the ML community. In the next sections, we present the main initiatives for monitoring the ML safety at runtime. We categorize these approaches into three categories, that are roughly the monitoring of the system’s intended usage conditions (Section 2.2 ), the detection of statistically anomalous input data (Section 2.3 ), and the detection of input data that lead to system failures (Section 2.4 ). 2.2 Monitoring the Operational Design Domain (ODD) The concept of Operational Design Domain (ODD) emerged first in the automotive safety domain as the set of conditions under which an automated driving system is intended to safely operate [ 39 ] . These conditions typically cover a broad range of constraints, including environmental factors, geographic restrictions, and temporal or situational boundaries. In aviation, the EASA has adapted the ODD concept in the context of ML integration in broader systems, defining it as “ the operating conditions under which a given AI/ML constituent is specifically designed to function as intended, including but not limited to environmental, geographical, and/or time-of-day restrictions ” [ 14 ] . Monitoring the ODD does not focus on safety conditions like traditional safety monitoring. It concerns the definition of boundaries of the system’s functionality, with the rationale that, out of these boundaries, the system’s functionality performances might decrease (without necessarily leading to a safety