Mitigating Distribution Shift in Graph-Based Android Malware Classification via Function Metadata and LLM Embeddings
Ngoc N. Tran, Anwar Said, Waseem Abbas, Tyler Derr, Xenofon D. Koutsoukos · 2025
Read Mitigating Distribution Shift in Graph-Based Android Malware Classification via Function Metadata and LLM Embeddings by Ngoc N. Tran, Anwar Said, Waseem Abbas, Tyler Derr, Xenofon D. Koutsoukos (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-06734.txt.
Introduction
Ngoc Tran and four co-authors test how Android malware detectors hold up in the wild in a 2025 paper. Graph-based classifiers, which read an app's function-call web, boast accuracy above ninety-four percent on standard tests. Yet on fresh variants of known malware families, their accuracy can fall by nearly half, a drop the authors call a critical, overlooked risk for real deployment.
The team built a benchmark suite that mimics two real shifts: same families in new dress, and wholly new domains. They also finger a flaw in the usual setup: the graphs carry structure only, throwing away the meaning inside each function. Their fix enriches the graphs with light function metadata and language-model embeddings of the code. Tests confirmed the enriched view reasons better across shifts, the start of the paper's case for data-centered defenses.
Catalog record
| CREATOR | Ngoc N. Tran, Anwar Said, Waseem Abbas, Tyler Derr, Xenofon D. Koutsoukos |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 56 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2508-06734
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Mitigating Distribution Shift in Graph-Based Android Malware Classification via Function Metadata and LLM Embeddings Ngoc N. Tran, Anwar Said, Waseem Abbas, Tyler Derr, Xenofon D. Koutsoukos Abstract Graph-based malware classifiers can achieve over 94% accuracy on standard Android datasets, yet we find they suffer accuracy drops of up to 45% when evaluated on previously unseen malware variants from the same family—a scenario where strong generalization would typically be expected. This highlights a key limitation in existing approaches: both the model architectures and their structure-only representations often fail to capture deeper semantic patterns. In this work, we propose a robust semantic enrichment framework that enhances function call graphs with contextual features, including function-level metadata and, when available, code embeddings derived from large language models. The framework is designed to operate under real-world constraints where feature availability is inconsistent, and supports flexible integration of semantic signals. To evaluate generalization under realistic domain and temporal shifts, we introduce two new benchmarks: MalNet-Tiny-Common and MalNet-Tiny-Distinct, constructed using malware family partitioning to simulate cross-family generalization and evolving threat behavior. Experiments across multiple graph neural network backbones show that our method improves classification performance by up to 8% under distribution shift and consistently enhances robustness when integrated with adaptation-based methods. These results offer a practical path toward building resilient malware detection systems in evolving threat environments. Introduction Android malware continues to evolve rapidly, posing persistent challenges to the reliability and robustness of automated detection systems. Recent advances in graph-based learning have led to promising approaches for malware classification by representing applications as function call graphs (FCGs), where nodes correspond to individual functions and directed edges represent invocation relationships ( Yang et al. 2021 ; Harang and Rudd 2020 ; Joyce et al. 2021 ) . Graph Neural Networks (GNNs) ( Kipf and Welling 2017 ; Hamilton, Ying, and Leskovec 2018 ) applied to these graphs enable the modeling of structural and behavioral patterns that are indicative of malicious behavior ( Li et al. 2022 ) . Compared to image-based approaches like byteplot representations ( Freitas, Duggal, and Chau 2022 ) , graph-based methods provide a more interpretable, semantically structured, and functionally meaningful abstraction. Graph-based malware classification gained momentum with the release of MalNet ( Freitas et al. 2021 ) , a large-scale dataset comprising over 1.5 million Android malware samples represented as FCGs. A smaller, balanced subset known as MalNet-Tiny was released to support benchmarking. Since then, MalNet-Tiny has become a standard testbed for evaluating GNN architectures, with state-of-the-art models achieving over 94% accuracy ( Rampášek et al. 2023 ; Shirzad et al. 2023 ) . However, these results often assume standard data splits. Recent work ( Wu et al. 2024 ) shows that performance drops significantly when models are evaluated on samples from malware families not seen during training, highlighting a critical weakness under distribution shift. A key limitation underlying this brittleness is the lack of semantic information in MalNet-Tiny. During graph construction, all function-level metadata—such as names, types, and code—was removed to avoid exposing potentially sensitive artifacts ( Freitas et al. 2021 ) . While this decision aimed to reduce reverse-engineering risks, we argue it unnecessarily restricts the model’s ability to reason about functional behavior. First, the full malware binaries used to generate these graphs are publicly available ( Allix et al. 2016 ; Freitas, Duggal, and Chau 2022 ) . Second, prior work has shown that semantic features can be integrated without compromising security ( Anderson and Roth 2018 ; Yang et al. 2021 ) . Moreover, we hypothesize that this omission impairs generalization for FCGs, whose edges denote information flow rather than structural similarity. In such graphs, node semantics are essential for effective message passing and representation learning ( Hamilton, Ying, and Leskovec 2018 ; Hu et al. 2020 ) . Without them, models tend to memorize local structures that do not transfer well across malware families. Motivated by this insight, we propose an enhanced attributed graph construction framework tailored for Android malware classification under distribution shift. Our method enriches each function call graph (FCG) with semantic node features extracted directly from the malware’s code structure. Specifically, we extract a set of lightweight, widely available metadata features—including function names, method signatures, access flags, instruction statistics, and Android-specific behaviors—commonly used in malware analysis ( Anderson and Roth 2018 ; Yang et al. 2021 ) . When decompiled source code is available, we embed function bodies using a large language model (LLM), providing dense representations of behavioral semantics. These semantic vectors are combined with standard structural features and injected as node attributes for downstream graph neural networks (GNNs). Importantly, our framework is designed for real-world deployment scenarios, where semantic features may be partially missing. To address this, we introduce three collation strategies—Trim, Zero, and Prune—that transform partially defined graphs into a consistent format suitable for learning. To evaluate robustness under distribution shifts, we construct and release two new benchmark variants of MalNet-Tiny. In MalNet-Tiny-Common , training and test samples are drawn from overlapping malware families, while in MalNet-Tiny-Distinct , test samples originate from families unseen during training ( Wu et al. 2024 ) . Across multiple GNN backbones, we find that our proposed semantic graph construction significantly improves classification accuracy under both settings. These results highlight the critical role of semantic information in improving the generalization of graph-based malware classifiers in realistic settings. Our key contributions are as follows: • We identify and empirically demonstrate the brittleness of state-of-the-art graph-based Android malware classifiers under distribution shift, where accuracy drops sharply on test samples drawn from previously unseen malware families. • We propose a semantic feature enrichment framework for Android function call graphs that augments structural graphs with interpretable metadata and LLM-derived embeddings. To address real-world data quality challenges—where semantic features may be inconsistently available—we introduce three collation strategies that robustly transform incomplete graphs into usable inputs for graph neural networks. • We construct and release two new benchmark datasets, MalNet-Tiny-Common and MalNet-Tiny-Distinct , designed to evaluate classifier robustness under intra-family and cross-family distribution shifts. • Across multiple GNN architectures, we empirically show that our approach significantly improves classification performance under distribution shift and boosts robustness when combined with adaptation-based training. Preliminaries MalNet and Graph-Based Malware Data MalNet ( Freitas et al. 2021 ) is a large-scale dataset of Android malware samples, where each sample is represented as a FCG extracted using AndroGuard ( Desnos and Gueguen 2018 ) . Each node in the FCG corresponds to a function, and edges indicate function calls. Labels are derived from VirusTotal reports and unified into a hierarchy of malware families and types using Euphony ( Hurier 2025 ) . A cleaned and balanced subset, MalNet-Tiny, was introduced to support manageable training and evaluation, with each class defined by a unique (family, type) pair. A more detailed discussion on MalNet and an example of this labeling structure is shown in Appendix. Distribution Shift in Malware Classification We consider a standard supervised classification setting, where each malware sample is denoted by X , and its associated label by Y . The training data is drawn from a source distribution \mathcal{D} , while test-time samples may be drawn from a different target distribution \mathcal{D}^{\prime} . Distribution shift then refers to