A Subjective Logic-based method for runtime confidence updates in safety arguments Conference: The 41st ACM/SIGAPP Symposium on Applied Computing; March 23–27, 2026; Thessaloniki, Greece The 41st ACM/SIGAPP Symposium on Applied Computing (SAC ’26), March 23–27, 2026, Thessaloniki, Greece DOI: 10.1145/3748522.3779865 ISBN: 979-8-4007-2294-3/2026/03 CCS: Theory of computation Automated reasoning CCS: Computing methodologies Uncertainty quantification CCS: Software and its engineering Software safety
Benjamin Herd, Jessica Kelly, Clarissa Heinemann, João-Vitor Zacchi · 2026
Read A Subjective Logic-based method for runtime confidence updates in safety arguments Conference: The 41st ACM/SIGAPP Symposium on Applied Computing; March 23–27, 2026; Thessaloniki, Greece The 41st ACM/SIGAPP Symposium on Applied Computing (SAC ’26), March 23–27, 2026, Thessaloniki, Greece DOI: 10.1145/3748522.3779865 ISBN: 979-8-4007-2294-3/2026/03 CCS: Theory of computation Automated reasoning CCS: Computing methodologies Uncertainty quantification CCS: Software and its engineering Software safety by Benjamin Herd, Jessica Kelly, Clarissa Heinemann, João-Vitor Zacchi (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-22530.txt.
Introduction
Herd and team present a method for live safety cases, adding live trust changes. Trust is scored across the project by mixing design-time data and recent live SPIs in one Subjective Logic case. SPI data is checked nonstop. Set claims update by a rule: trust rises without breaches and falls fast after them. The design prizes safety-first speed over exact old Bayesian after-the-fact updates. The team shows the method on a mock roadwork zone assist with an ML cone detector, showing trust shifting.
Catalog record
| CREATOR | Benjamin Herd, Jessica Kelly, Clarissa Heinemann, João-Vitor Zacchi |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 40690 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.22530
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
A Subjective Logic-based method for runtime confidence updates in safety arguments Conference: The 41st ACM/SIGAPP Symposium on Applied Computing; March 23–27, 2026; Thessaloniki, Greece The 41st ACM/SIGAPP Symposium on Applied Computing (SAC ’26), March 23–27, 2026, Thessaloniki, Greece DOI: 10.1145/3748522.3779865 ISBN: 979-8-4007-2294-3/2026/03 CCS: Theory of computation Automated reasoning CCS: Computing methodologies Uncertainty quantification CCS: Software and its engineering Software safety Benjamin Herd, Jessica Kelly, Clarissa Heinemann, João-Vitor Zacchi Abstract . We present a method for dynamic quantitative assurance that enhances static safety cases with continuous, runtime-driven confidence updates. The method quantifies and propagates confidence across the development lifecycle by integrating design-time evidence and windowed runtime Safety Performance Indicators (SPIs) within a single Subjective Logic (SL)-based assurance case. At runtime, SPI evidence is continuously evaluated, and targeted claims are updated using a rule that increases confidence in the absence of violations and imposes prompt penalties when violations occur. This design prioritizes safety-relevant responsiveness over exact classical Bayesian posterior updates. We demonstrate the method using a simulation-based construction zone assist function, focusing on an ML-based construction cone detection component, and show how confidence evolves as SPI evidence is observed in operation. 1. Introduction Safety assurance arguments for machine learning (ML) components offer structured justifications that a system is safe to operate within a defined operational context. As these systems are deployed in complex and dynamic environments, there is a need to reflect the evolving confidence in the validity of the assurance argument over time. According to UL 4600 1 , an argument is valid if (1) all claims are supported by evidence, and (2) the evidence itself is valid, i.e. supported by factual, objective data. However, given the finite amount of training and testing data and the incomplete understanding of the operational context including potential triggering conditions, there is always a chance of the argument becoming invalidated over time. Shifts in data distribution can lead to model drift and challenge the continuing relevance of the initial argument; assumptions about the surrounding system and the operational context may be invalidated by evolving requirements or the integration of new technologies. It is thus crucial to adopt a systematic approach for ongoing reassessment of assurance arguments and to ensure sustained confidence and alignment with operational demands. One way of incorporating runtime information into assurance arguments is through Safety Performance Indicators (SPIs) . SPIs are attached to claims in the argument and defined as “metrics supported by evidence that use threshold comparisons of condition claims in a safety case” 1 . The metric values are empirically gathered by observing the system throughout its lifecycle. In this paper, we extend prior work 12 13 on the formulation of static, quantitative assurance arguments using Subjective Logic (SL) by incorporating runtime information for the purpose of continuous assurance . More precisely, we integrate the concept of SPIs into quantitative SL-based assurance arguments to study the evolution of confidence in the assurance case, enabling a more dynamic and responsive approach to safety validation. The approach allows for runtime evidence to target specific claims and propagate through the same formalism (SL) used for computing assurance confidence in the static argument. We deliberately go beyond classical probabilistic or Bayesian update by proposing an update rule that increases confidence when no SPI violations occur and incurs an immediate penalty in the case of violations. The approach thus prioritizes safety-relevant responsiveness over exact conjugate-posterior equivalence in a purely Bayesian setting. Our contribution is an SL-native, explicitly non-Bayesian update method that integrates windowed runtime SPIs into quantitative assurance by choosing SL operators which enable gradual confidence accumulation in the absence of violations and immediate penalties when violations occur. We demonstrate the method using an ML-based construction cone detection simulation, illustrating the evolution of first- and second-order confidence in the argument during operational use. The paper is structured as follows: Section 2 provides background information on assurance confidence estimation, SPIs, and SL. Section 3 introduces an SL-based method for modelling confidence propagation from SPIs to claims in dynamic assurance arguments and illustrates it with a simulation of ML-based construction cone detection. Related work is presented in Section 4 . The paper concludes with a discussion in Section 5 . 2. Background 2.1. Assurance confidence estimation According to ISO/IEC 15026, an assurance argument is “a reasoned, auditable artifact that supports the contention that its top-level claim is satisfied, including systematic arguments and its underlying evidence and explicit assumptions that support the claim(s)” 15 . Assurance arguments are used to demonstrate that safety-critical systems satisfy their safety requirements in a given context. Assurance confidence assessments determine how much trust can be placed in a given argument. Various confidence assessment methods have been proposed and can be broadly classified into qualitative and quantitative approaches. Qualitative methods typically rely on expert judgment and involve dialectical techniques such as defeaters , i.e. attacks that question the validity of specific claims or evidence within the argument. Defeaters play a key role in identifying potential weaknesses, gaps in reasoning, or uncertainties that could compromise the overall strength of safety assurance. Qualitative defeaters exert a binary effect on their target: they either refute it or leave it unchanged. On the other hand, quantitative methods for assurance confidence estimation aim to deliver numerical valuations of confidence through the use of mathematical and statistical approaches, e.g. Baconian probabilities 8 ; 9 , Bayesian inference 10 ; 3 ; 14 , Dempster-Shafer belief functions 2 ; 21 , or SL 7 ; 23 ; 12 . As discussed below, the nature of confidence assessment influences the representation of SPIs and their impact on the underlying claims. 2.2. Safety Performance Indicators (SPIs) ML-based systems are often deployed in unpredictable environments which cannot be completely specified at design time. There is thus a need to continuously reassess the validity of the argument throughout the lifecycle. One way of incorporating runtime information into assurance arguments is through the use of Safety-Performance Indicators (SPIs), a concept first introduced in the context of autonomous vehicles by the UL 4600 standard 1 . UL 4600 makes a distinction between traditional key performance indicators (KPIs) and SPIs, which are defined as performance metrics specifically related to safety . SPIs are defined to be “metrics supported by evidence that use threshold comparisons of condition claims in a safety case”. These metric values are obtained empirically by monitoring the system over its lifecycle. Each SPI is associated with a specific claim in the safety case, and if an SPI’s threshold is violated (meaning the SPI is false), it indicates a violation of the claim. The standard further distinguishes between lagging and leading SPIs. Lagging SPIs are inherently reactive and might include metrics such as the number of accidents, instances of system failures, or the frequency of interventions by human drivers. On the other hand, leading SPIs focus on proactive measures such as the number of low-confidence classifications. Integrating both lagging and leading SPIs allows for a balanced approach to safety management. SPIs may impact confidence in the associated claim(s) both positively and negatively. A lack of SPI violations should, intuitively, strengthen confidence in a claim. The amount to which this impact can be made explicit in the argument relies on the underlying confidence assessment method. For