Example-Based Concept Analysis Framework for Deep Weather Forecast Models
Soyeon Kim, \aff a Junho Choi, \aff a Subeen Lee, \aff a, Jaesik Choi, \aff a,b · 2025
Read Example-Based Concept Analysis Framework for Deep Weather Forecast Models by Soyeon Kim, \aff a Junho Choi, \aff a Subeen Lee, \aff a, Jaesik Choi, \aff a,b (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-00831.txt.
Introduction
Soyeon Kim and co-authors build an explanation tool for AI weather forecasters that argues by example. Their 2025 paper notes that forecast models are trusted with high-stakes decisions, yet their reasoning stays opaque. Most explanation methods are built for AI specialists rather than the meteorologists who must defend a forecast.
Their framework finds past cases the model processed in the same way as the target forecast, and presents them as visual and conceptual analogues. Concept assignments come with probabilities, so ambiguous weather mechanisms are flagged rather than hidden. A human-annotated concept dataset bridges the model's internal vectors and expert language, and a user interface was developed with domain experts in the loop. The design philosophy is the contribution: explanations are a user interface problem, and the user is a working forecaster, not a machine learning engineer.
Catalog record
| CREATOR | Soyeon Kim, \aff a Junho Choi, \aff a Subeen Lee, \aff a, Jaesik Choi, \aff a,b |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 59 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2504-00831
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Example-Based Concept Analysis Framework for Deep Weather Forecast Models Soyeon Kim, \aff a Junho Choi, \aff a Subeen Lee, \aff a, Jaesik Choi, \aff a,b Abstract To improve the trustworthiness of an AI model, finding consistent, understandable representations of its inference process is essential. This understanding is particularly important in high-stakes operations such as weather forecasting, where the identification of underlying meteorological mechanisms is as critical as the accuracy of the predictions. Despite the growing literature that addresses this issue through explainable AI, the applicability of their solutions is often limited due to their AI-centric development. To fill this gap, we follow a user-centric process to develop an example-based concept analysis framework, which identifies cases that follow a similar inference process as the target instance in a target model and presents them in a user-comprehensible format. Our framework provides the users with visually and conceptually analogous examples, including the probability of concept assignment to resolve ambiguities in weather mechanisms. To bridge the gap between vector representations identified from models and human-understandable explanations, we compile a human-annotated concept dataset and implement a user interface to assist domain experts involved in the the framework development. \statement This study investigates deep neural networks’ (DNNs) ability to encode semantic patterns of precipitation mechanisms and aims to provide a ready-to-deploy explainable artificial intelligence (XAI) tool. Key findings reveal that DNNs can extract nonlinear precipitation mechanisms and represent semantically meaningful meteorological attributes. The concept explanations align with expert perceptions, enhancing the interpretability and trustworthiness of model predictions. These findings demonstrate DNNs’ potential to provide insightful, explainable predictions in meteorology, improving the trustworthiness of DNNs for practitioners. Follow-up research could involve refining the XAI framework, exploring its application for other meteorological phenomena, regions or scales, and integrating it with operational systems to assess the strengths and limitations in real-world scenarios. 1 Introduction Recent applications of deep neural networks (DNNs) in meteorology demonstrate superior predictive performance and computation cost compared to traditional numerical weather prediction (NWP) models ( Bi et al., 2023 ; Lam et al., 2023 ; Tang and Zhang, 2022 ) . However, actual adoption of DNNs in operational forecasting is slow due to their black-box nature: the high stakes associated with incorrect predictions require practitioners to have an intimate understanding of the inference process, an aspect that typical DNNs cannot address. A significant number of recent studies attempt to resolve this issue through explainable artificial intelligence (XAI) ( Yang et al., 2024 ; Kim et al., 2023 ; Toms et al., 2020 ; McGovern et al., 2019 ; Gagne II et al., 2019 ) . Existing applications of XAI in meteorology are often developed from the perspective of AI experts, reducing the utility of explanations for domain users. One solution to this problem is collaborating with the user population, which has been shown to offer significant benefits ( Ravuri et al., 2021 ) . This study builds upon this notion by constructing a user-centric XAI framework with an experts-in-the-loop approach, cooperating with experts at the Korea Meteorological Agency (KMA) and the National Institute of Meteorological Sciences (NIMS). Given that typical XAI methods are difficult for humans to comprehend ( Kim et al., 2023 ) , we incorporate example-based explanation (explaining through samples that satisfy some criteria) and concept explanation (explaining through human-understandable semantics). We also design a user interface to enhance the suitability of the framework for real applications, and perform case studies to measure the alignment between the generated explanations and domain knowledge. For the explanation task, we tackle the question whether DNNs’ representations encode semantically meaningful nonlinear precipitation mechanisms, a task that is yet to be addressed in prior literature ( Kurihana et al., 2024 ; Jo et al., 2020 ; Park et al., 2021 ) . Specifically, we address the following two topics: Can we detect nonlinear precipitation mechanisms from trained DNNs? Can we identify the presence of meaningful meteorological attributes such as convectional, frontal, orographic, and convergence mechanisms from internal representation space in trained DNNs? The rest of this paper is organized as follows. Section 2 provides an overview of past literature on example-based, concept-based, and user-centric explanations. Section 3 outlines the example-based concept analysis framework, human annotation process for meteorological data, and user interface design. Section 4 discusses the experimental setup, including model and data. Section 5 assesses the results of the experiments both quantitatively and qualitatively before discussing the implications of the findings. Section 6 concludes the paper. 2 Related Work 2.1 Example-Based Explanation Example-based explanation is a popular XAI method that is easy for layman users to understand, a characteristic essential to user-centric XAI ( Molnar, 2020 ) . Nearest neighbor search is one such method, but the results can vary by the proximity metric ( Johnson et al., 2016 ) . Euclidean distance in the feature space of DNN is more human perceptually close than sophisticated similarity measures in the input level ( Zhang et al., 2018 ; Amir and Weiss, 2021 ) . A recent study demonstrates that configuration distance - the Hamming distance between activation status of feature vectors - is also semantically aligned with human perception ( Chang et al., 2024 ) . However, since all metrics inevitably require computing pairwise distances, nearest neighbor search does not scale to high-dimensional data. To address this issue, we implement a nearest neighbor search with dimensionality reduction. 2.2 Concept Explanation A concept refers to semantic representations such as objects, shapes, textures, and colors ( Schwalbe, 2022 ) . Concept analysis is applied in numerous domains since it is intrinsically human-intelligible ( Schut et al., 2023 ; Cai et al., 2019 ) . Another advantage of concept explanation is the targeted concepts does not need to be intrinsic to the target model’s task: most studies use human annotations to assign meaningful concepts ( Kim et al., 2018 ) , which may not necessarily be directly associated with class labels. Therefore, we can probe the high-level semantic information from the internal representations of the AI models. In meteorology, concept analysis has been studied to identify weather mechanisms captured in AI models, such as the eye of the typhoon in a DNN ( Sprague et al., 2019 ) . However, to the best of our knowledge, representations captured in spatiotemporal models has not yet been studied in previous papers. Our work builds upon Testing with Concept Activation Vectors (TCAV) ( Kim et al., 2018 ) to identify spatiotemporal patterns for rainfall mechanisms. 2.3 Concept Prober Probing is a technique to understand the concepts captured in trained models ( Alain and Bengio, 2016 ) and the influence of these representations on model prediction ( Belinkov, 2022 ) . Approaches include using (a) the probability from independently trained support vector machine (SVM) classifiers for each concept ( Kim et al., 2018 ) , (b) mutual information between representations and target labels ( Pimentel et al., 2020 ) , and (c) Gaussian processes ( Wang et al., 2024 ) . Probing is applied in tasks such as finding factual associations of large language models ( Meng et al., 2022 ) , analyzing causality perspective ( Vig et al., 2020 ) , validating model hallucination ( Azaria and Mitchell, 2023 ) , or identifying