Bridging Distribution Shift and AI Safety: Conceptual and Methodological Synergies
Chenruo Liu, Kenan Tang, Yao Qin, Qi Lei · 2025
Read Bridging Distribution Shift and AI Safety: Conceptual and Methodological Synergies by Chenruo Liu, Kenan Tang, Yao Qin, Qi Lei (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-22829.txt.
Introduction
Chenruo Liu and co-authors link two AI fields that grew up apart: data shift and AI safety in a 2025 paper. Data shift is what happens when the world a system meets drifts from the world it trained on. Safety covers the harms such systems can cause or let through.
The paper draws two kinds of ties. In one, a fix for a named kind of shift also serves a named safety goal. In the other, a shift snag and a safety snag turn out to be the same shape, so tools cross over both ways. The unified view aims to pull the two fields into joint work.
Catalog record
| CREATOR | Chenruo Liu, Kenan Tang, Yao Qin, Qi Lei |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 98 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-22829
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Bridging Distribution Shift and AI Safety: Conceptual and Methodological Synergies Chenruo Liu, Kenan Tang, Yao Qin, Qi Lei Abstract . This paper bridges distribution shift and AI safety through a comprehensive analysis of their conceptual and methodological synergies. While prior discussions often focus on narrow cases or informal analogies, we establish two types of connections between specific causes of distribution shift and fine-grained AI safety issues: (1) methods addressing a specific shift type can help achieve corresponding safety goals, or (2) certain shifts and safety issues can be formally reduced to each other, enabling mutual adaptation of their methods. Our findings provide a unified perspective that encourages deeper integration between distribution shift and AI safety research. CCS: General and reference Surveys and overviews CCS: Computing methodologies Machine learning 1. Introduction Machine learning (ML) models have achieved remarkable success across complex tasks in various fields, including high-stakes applications such as autonomous driving, healthcare, finance, and legal domains ( Grigorescu et al., 2020 ; Shailaja et al., 2018 ; Dixon et al., 2020 ; Surden, 2021 ) . Traditionally, ML models are trained on historically collected data and subsequently deployed in a test domain. While these ML models succeed in the case where the training and testing data share the same distribution, performance drops are expected when distribution shifts from training to test domains ( Koh et al., 2021 ; Cai et al., 2025 ) . Beyond performance concerns, the failure of ML models to robustly handle distribution shifts in real-world settings can lead to severe safety issues, such as malfunctions in autonomous driving systems ( Stoler et al., 2024 ) and errors in clinical systems ( Ma et al., 2022 ) . In more recent examples, despite substantial efforts in alignment research, large language models (LLMs) and large multimodal models still suffer from outputting inappropriate content under distribution shift ( Wang et al., 2023a ; Jeong et al., 2025 ) . As artificial intelligence (AI) systems continue to evolve, minimizing their safety risks has emerged as a critical priority ( Amodei et al., 2016 ) . Consequently, distribution shift, which is inherently linked to AI safety, has become a central research challenge. A deep understanding of the relationship between distribution shift and AI safety is undoubtedly beneficial for the safe deployment of ML models in real-world applications. However, although previous studies have frequently mentioned the connection between these two fields ( Amodei et al., 2016 ; Li et al., 2023a ; Ji et al., 2023 ; Lu et al., 2025 ; Fang et al., 2025 ) , they exhibit several limitations. First, they focus narrowly on the relationship between distribution shift and a single aspect of AI safety (e.g., robustness ( Amodei et al., 2016 ) , trustworthiness ( Fang et al., 2025 ; Li et al., 2023a ) , or alignment ( Lu et al., 2025 ; Ji et al., 2023 ) ) but lack a more comprehensive framework. Second, these studies often treat both distribution shift and the targeted AI safety aspect as general concepts, rather than breaking them down into specific categories ( Amodei et al., 2016 ) . Third, existing work discusses the connection between distribution shift and AI safety as an informal analogy without identifying the mathematical relationships among their rigorous definitions. As a result, there is still a lack of detailed exploration into the connections between specific topics in distribution shift and AI safety. {forest} {forest} Figure 1 . Connections between distribution shift causes and AI safety issues. In each leaf node, the cited definition number indicates where the concept is formally defined, and the section number in parentheses indicates the section where the connection is discussed in detail. One-directional arrows: Methods addressing a specific cause of distribution shift can also achieve a particular safety goal. Bidirectional solid arrows: Due to inherent consistency between a certain distribution shift cause and a corresponding safety issue, the two can be reduced from one another and methods derived for them can be mutually adapted. For one-directional connections, we list five representative references under the corresponding distribution shift cause. For bidirectional connections, we list five representative references under both the distribution shift cause and the associated safety issue. Although environmental change is linked to multiple safety issues, we show only five representative references here. More can be found in the corresponding sections. In this paper, we offer the first comprehensive, one-to-one aligned, and mathematically grounded analysis of the relationship between distribution shift and AI safety (Figure 1 ). On the one hand, according to the underlying causes, we examine distribution shifts from three main categories: selection bias ( Shimodaira, 2000 ; Santurkar et al., 2021 ) , spurious correlation ( Simon, 1954 ; Ye et al., 2024 ) , and label shift ( Lipton et al., 2018 ; Tachet des Combes et al., 2020 ; Garg et al., 2022 ) and six more fine-grained categories. On the other hand, we focus on the following categories of AI safety issues: AI security ( Sarker et al., 2021 ) , AI fairness ( Zhou et al., 2020 ) , AI trustworthiness ( Kaur et al., 2022 ) , and AI democracy ( Seger et al., 2023 ) . For each specific topic discussed within these main categories, we provide its rigorous definition. Next, we systematically identify connections between specific distribution shift causes and concrete AI safety issues by analyzing the underlying mathematical relationships between their definitions. Based on the identified connections, we then review the relevant literature to establish methodological links between these specific topics. Our analysis exhibits three characteristics that distinguish it from previous work: • Comprehensive : We cover a diverse range of aspects of distribution shifts and AI safety, providing a more comprehensive discussion of the relationship between these two fields than previous specialized surveys. • One-to-one aligned : We directly align each specific distribution shift cause with relevant concrete AI safety issues to enable a clear, one-to-one comparison. The comparison leads to actionable future directions. • Mathematically grounded : The connections we identify are rooted in rigorous mathematical relationships, as shown in the “Definition-Level Connection” boxes, which guide our subsequent review of the literature. Building on these features, our work reveals two key relationships between distribution shifts and AI safety. First, methods addressing a specific cause of distribution shift can achieve a concrete AI safety goal (indicated by one-directional arrows in Figure 1 ). Second, methods designed for a specific distribution shift cause and a concrete AI safety issue can be mutually adapted because of their definitional consistency (indicated by bidirectional solid arrows in Figure 1 ). These two types of connections highlight shared research interests and create opportunities for methodological exchange across distribution shift and AI safety. Our findings establish a more unified perspective on distribution shifts and AI safety, fostering closer collaboration between these two research communities. The remainder of this paper is organized as follows. Section 2 introduces notations used throughout the paper and provides a topic overview. In Sections 3 , 4 , and 5 , we examine how the specific causes of distribution shift within the main categories are connected to various concrete AI safety issues. Finally, Section 6 concludes the paper.