Concept-based explanations of Segmentation and Detection models in Natural Disaster Management
Samar Heydari, Jawher Said, Galip Ümit Yolcu, Evgenii Kortukov, Elena Golimblevskaia, Evgenios Vlachos, Vasileios Mygdalis, Ioannis Pitas, Sebastian Lapuschkin, Leila Arras · 2026
Read Concept-based explanations of Segmentation and Detection models in Natural Disaster Management by Samar Heydari, Jawher Said, Galip Ümit Yolcu, Evgenii Kortukov, Elena Golimblevskaia, Evgenios Vlachos, Vasileios Mygdalis, Ioannis Pitas, Sebastian Lapuschkin, Leila Arras (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-23020.txt.
Introduction
Heydari and co-authors build an explainability framework for disaster response models. Their models segment floods and detect cars, often on drone hardware. Emergency teams need to trust the outputs, which black-box models hinder. The framework extends relevance propagation through gated fusion layers in PIDNet. A concept-based method then gives local and global explanations in learned concepts. Tests on a public flood dataset give reliable, readable explanations. Inference stays near real time, suiting small drone platforms.
Catalog record
| CREATOR | Samar Heydari, Jawher Said, Galip Ümit Yolcu, Evgenii Kortukov, Elena Golimblevskaia, Evgenios Vlachos, Vasileios Mygdalis, Ioannis Pitas, Sebastian Lapuschkin, Leila Arras |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 18688 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.23020
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Concept-based explanations of Segmentation and Detection models in Natural Disaster Management Samar Heydari, Jawher Said, Galip Ümit Yolcu, Evgenii Kortukov, Elena Golimblevskaia, Evgenios Vlachos, Vasileios Mygdalis, Ioannis Pitas, Sebastian Lapuschkin, Leila Arras Abstract Deep learning models for flood and wildfire segmentation and object detection enable precise, real-time disaster localization when deployed on embedded drone platforms. However, in natural disaster management, the lack of transparency in their decision-making process hinders human trust required for emergency response. To address this, we present an explainability framework for understanding flood segmentation and car detection predictions on the widely used PIDNet and YOLO architectures. More specifically, we introduce a novel redistribution strategy that extends Layer-wise Relevance Propagation (LRP) explanations for sigmoid-gated element-wise fusion layers. This extension allows LRP relevances to flow through the fusion modules of PIDNet, covering the entire computation graph back to the input image. Furthermore, we apply Prototypical Concept-based Explanations (PCX) to provide both local and global explanations at the concept level, revealing which learned features drive the segmentation and detection of specific disaster semantic classes. Experiments on a publicly available flood dataset show that our framework provides reliable and interpretable explanations while maintaining near real-time inference capabilities, rendering it suitable for deployment on resource-constrained platforms, such as Unmanned Aerial Vehicles (UAVs). [orcid=0009-0002-2247-2124 ] [orcid=0000-0001-5473-5262 ] [orcid=0009-0006-7555-8641 ] [orcid=0000-0002-0762-7258, [email protected] ] *1 [[email protected] ] *1 keywords Natural Disaster Management ,Semantic Segmentation ,Object Detection ,Concept-based Explanations. † † copyrightyear: 2026 † † copyright: Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). † † venue: Preprint † † address: Department of Artificial Intelligence, Fraunhofer Heinrich Hertz Institute, Berlin, Germany † † address: Department of Informatics, Aristotle University of Thessaloniki, Thessaloniki, Greece † † corresp: Corresponding author. 1 Introduction Natural disasters such as flood and wildfire pose a serious threat to public safety and the global economy 25 , requiring advanced monitoring systems. Deep neural networks (DNNs) for the semantic segmentation and the detection of objects from drone and satellite images can provide crucial information for situational awareness. However, their “black-box” nature limits the trust and confidence required for emergency responses. We address this challenge by introducing an end-to-end explainable framework for DNN-based segmentation and object detection in natural disaster management (NDM) building upon Layer-wise Relevance Propagation (LRP) 10 , an explainable AI attribution technique that provides feature-level importance scores (aka relevances) to explain individual predictions of neural networks. Together with its concept-based extensions, Concept Relevance Propagation (CRP) 1 ; 12 and Prototypical Concept-based Explanations (PCX) 11 , LRP provides an understanding of DNN model behavior both globally (i.e., dataset-wide) and in terms of human-comprehensible concepts. LRP and its concept-based extensions already demonstrated their usefulness in computer vision, but they have not yet been applied to a PIDNet architecture nor to the NDM domain. Additionally, Prototypical Concept-based Explanations (PCX) 11 were so far confined to image classification models: we present its first extension to semantic segmentation and object detection models. Contributions. In summary, our novel contributions are as follows: • We apply LRP 10 and concept-based explanations 1 ; 12 for the first time in NDM. • We extend LRP 10 to a PIDNet architecture. • We apply PCX 11 for the first time on segmentation and detection models. 2 Related Work 2.1 Deep learning for Natural Disaster Management Deep learning methods are increasingly common in Natural Disaster Management (NDM) 2 ; 19 for real-time monitoring and response planning. In the context of wildfire, deep learning has been employed for satellite-based 3 and aerial-based image analysis 18 . Recent solutions leverage high-performance architectures like PIDNet 34 , with specialized extensions for fusing infrared and RGB spectra such as RoboFireFuseNet 13 , and Neural Architecture Search (NAS) methods 32 for optimizing the number of trainable parameters. Regarding flood segmentation, new learning processes utilize Self-Knowledge Distillation (Self-KD) to encode foreground information while suppressing background noise 22 . Core enabling resources include publicly available datasets for wildfire/burned area segmentation such as Blaze 28 , datasets and benchmarks for flood segmentation 14 , as well as synthetic data 6 ; 30 and weakly supervised learning frameworks 31 ; 5 to address data scarcity and improve model generalization. Recent efforts regarding object detection in NDM employed transfer learning on YOLO models 24 and visibility-enhanced models, such as VE-DINO 35 , to handle occlusion-prone environments. Other research focused on combining deep learning vision models with social media analysis 33 and sentiment analysis 4 to give first responders a holistic understanding of the situation, which can help them allocate resources and better predict the progression of the disaster. 2.2 Explainable AI Explainable AI (XAI) methods aim to achieve fidelity (explanations that align with the actual logic of the model) and comprehensibility for human operators. One popular class of XAI methods include local feature attributions that explain individual predictions by assigning importance scores to the input and latent features. These can be roughly divided into three groups: 1) perturbation/surrogate-based (e.g., SHAP 20 ), 2) gradient-based (e.g., Gradient and Grad-CAM 29 ; 27 ), and 3) decomposition-based methods (e.g., LRP 10 ). Group 1 and 2 present the advantage of being model-agnostic. While group 1 are computationally expensive, group 2 are cheap but typically noisy and prone to gradient shattering. Group 3 methods can be made as efficient as a gradient backward pass 9 , however they require a careful design for new types of neural network layers. Evaluation in controlled environments w.r.t. ground truths has demonstrated the superior quality of the latter group of methods 8 ; 21 . 2.3 Concept-based explanations Concept Relevance Propagation (CRP) 1 ; 12 breaks down decisions into human-interpretable concepts by considering feature maps inside a convolutional neural network (CNN) as semantic concept detectors and conditioning the LRP backward pass on these concepts, making it possible to visualize concept-conditioned heatmaps in the input space, as well as retrieve samples that maximize the relevance of a concept. Prototypical Concept-based Explanations (PCX) 11 generalize this to global XAI by summarizing the model’s prediction behavior dataset-wide into prototypes. Another concept-based explanation is Testing with Concept Activation Vectors (TCAV) 15 which measures the gradient-based sensitivity of the model’s latent features w.r.t. pre-defined concept directions which are obtained by training a linear classifier to separate latent features of samples with and without that concept. 3 Methods 3.1 Layer-wise Relevance Propagation (LRP) Layer-wise Relevance Propagation (LRP) 10 is a post-hoc, model-specific explanation technique that performs a conservative backward decomposition of the model’s prediction, for instance, an object detection score or a semantic segmentation logit, backward through all the layers of the network until the input. The distribution of the relevance is controlled by local propagation rules that preserve the total relevance at each layer, i.e., \sum_{i}R_{i}^{(l)}=\sum_{j}R_{j}^{(l+1)} , where R_{i}^{(l)} is the relevance of neuron i in layer l . The basic LRP \epsilon -rule for a linear layer with neurons i and j in consecutive layers is given by: R_{i}^{(l)}=\sum_{j}\frac{z_{ij}}{\sum_{k}z_{kj}+\epsilon\cdot\operatorname{sign}\left(\sum_{k}z_{kj}\right)}R_{j}^{(l+1)} (1) In this equation, z_{ij} is the contribution of neuron i to neuron j in the forward pass (typically equal to the neuron’s activation x_{i} multiplied by the connection weight w_{ij} , i.e., z_{ij}=x_{i}\cdot w_{ij} ), and \epsilon is a small numerical stabilizer.