Transparent Malware Detection With Granular Assembly Flow Explainability via Graph Neural Networks

Griffin Higgins, Roozbeh Razavi-Far, Senior Member, IEEE , Hossein Shokouhinejad, and Ali A. Ghorbani, Senior Member, IEEE · 2026

Read Transparent Malware Detection With Granular Assembly Flow Explainability via Graph Neural Networks by Griffin Higgins, Roozbeh Razavi-Far, Senior Member, IEEE , Hossein Shokouhinejad, and Ali A. Ghorbani, Senior Member, IEEE (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-14511.txt.

Introduction

Griffin Higgins, Roozbeh Razavi-Far, Hossein Shokouhinejad, and Ali A. Ghorbani build a malware detector in this 2026 paper. Malware keeps growing more sophisticated, threatening, and evasive. Detection systems must keep pace, and they must be transparent as well as strong. Users need to know why a file was flagged.

The team proposes the Assembly Flow Graph, or AFG. It represents the assembly flow of a binary executable as graph data, in comprehensive form. Graph neural networks then read that graph to detect malware. The design gives granular explainability at the assembly flow level. Readers get a detector whose verdicts come with a traceable path through the code.

Catalog record

CREATORGriffin Higgins, Roozbeh Razavi-Far, Senior Member, IEEE , Hossein Shokouhinejad, and Ali A. Ghorbani, Senior Member, IEEE
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE56 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2601-14511

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Transparent Malware Detection With Granular Assembly Flow Explainability via Graph Neural Networks G. Higgins, R. Razavi-Far, H. Shokouhinejad, and A. Ghorbani are with Canadian Institute for Cybersecurity, University of New Brunswick, Fredericton, Canada (e-mail: [email protected]). Griffin Higgins, Roozbeh Razavi-Far, Senior Member, IEEE , Hossein Shokouhinejad, and Ali A. Ghorbani, Senior Member, IEEE Abstract As malware continues to become increasingly sophisticated, threatening, and evasive, malware detection systems must keep pace and become equally intelligent, powerful, and transparent . In this paper, we propose Assembly Flow Graph (AFG) to comprehensively represent the assembly flow of a binary executable as graph data. Importantly, AFG can be used to extract granular explanations needed to increase transparency for malware detection using Graph Neural Networks (GNNs). However, since AFGs may be large in practice, we also propose a Meta-Coarsening approach to improve computational tractability via graph reduction. To evaluate our proposed approach we consider several novel and existing metrics to quantify the granularity and quality of explanations. Lastly, we also consider several hyperparameters in our proposed Meta-Coarsening approach that can be used to control the final explanation size. We evaluate our proposed approach using the CIC-DGG-2025 dataset. Our results indicate that our proposed AFG and Meta-Coarsening approach can provide both increased explainability and inference performance at certain coarsening levels. However, most importantly, to the best of our knowledge, we are the first to consider granular explainability in malware detection using GNNs. I Introduction Malware is software explicitly designed with malicious intent for the purposes of disrupting digital information and information systems. The use of machine learning (ML) and deep learning (DL) to automatically detect malware has proven extremely promising. This is especially true for graph learning-based methods that operate on graph data extracted from program flow [ 1 ] . However, since GNNs are not intrinsically explainable , explanation methods must be used to derive explanations related to a particular prediction. While different types of explanations are possible, extracting an explanatory subgraph related to graph classification (e.g., malware detection) is of special interest to improve model transparency. Such a subgraph can guide our understanding of which underlying phenomenon is influential to a given prediction (i.e., malicious behavior). Subgraph explanations can also help confirm that an underlying prediction, irrespective of correctness, is being made in a non-spurious manner. Most importantly, the task of malware detection also demands heightened transparency as the cost of false negative predictions or misleading explanations are extremely high. Building on this, it is also important to assess the explanation quality for a given subgraph. This can be viewed and measured in two fundamental ways. Specifically, an explanation subgraph can be either sufficient , where the extracted subgraph matches the prior prediction of the model on the original sample or necessary where the original sample with the subgraph removed generates a different prediction from the prior models prediction on the original sample. However, when evaluated together, an explanatory subgraph that is able to achieve both a highly necessary and sufficient explanation gives rise to a further notion of a highly characterized explanation [ 2 ] . This is similar to the way that F1 score provides a deeper understanding of classwise performance with respect to both precision and recall. Furthermore, it is important to also consider explanation granularity . While prior works have considered GNN malware explainability on CFGs, a key limitation is that they are ultimately unable to provide highly characterized instruction level subgraph explanations. This is not a limitation of GNNs or methods per se, but rather of the underlying CFG graph structure. In a CFG, nodes are represented as sequences of non-branching assembly instructions, formally referred to as basic blocks, that are directly connected to other nodes by edges that represent branching (e.g., jump instructions). However, since the instructions do not play an observable role in the CFG graph structure they therefore cannot be explained via GNNs. Additionally, since basic blocks may contain many assembly instructions it becomes ambiguous as to which instructions contribute to the explained behavior. By comparison, granular instruction level explanations are superior and needed since they can provide unambiguous explanations needed to enhance model transparency. While some related works do consider aspects of instruction level program flow via opcode graphs as Markov chains [ 3 , 4 , 5 , 6 , 7 , 8 , 9 ] , our work is distinct and novel since we consider the assembly instruction level flow of an entire program as a graph, that we term Assembly Flow Graph (AFG). In our work, we use AFG to improve detection performance and strongly characterize the predicted behavior of a sample through the use of GNNs. This approach is important because it can provide granular explanations to help understand and characterize malware behavior. This is especially useful for assisting malware analysis where the number of instructions are large. However, many challenges related to the large size and scale of AFGs make most aspects of applying GNNs (preprocessing, embedding, training, explaining, etc.) non-trivial. To address this, we apply coarsening, a graph reduction technique, to reduce both nodes and edges in the original graph to a prespecified size. Critically, we select coarsening since it aims to maintain structural similarity to the original graph while also providing nodewise backtrackability from reduced to original nodes. Nonetheless, applying coarsening directly on AFG remains computationally challenging. To alleviate this, we propose Meta-Coarsening, shown at a high level in Fig. 1 , where we first coarsen the CFG, (i.e., AFG meta-structure) to approximate coarsening AFG directly. Once a coarsened C-CFG, dataset is constructed a GNN model is then trained and an explainer is used to derive explanations. Here, the most important C-CFG explanation subgraph is then backtracked onto the underlying AFG. Afterwords, an entirely separate backtracked AFG (B-AFG) dataset is used to train a separate GNN model. An explainer is then used to derive the final B-AFG subgraph level explanation at the assembly instruction level . This approach has the advantage that any subgraph of the AFG can be arbitrarily examined under a varying level of coarsening and adjusted to fit the computational requirements or explanation expressiveness. To evaluate our proposed approach, we conduct experiments on the Dynamically Generated Graphs for Malware Analysis (CIC-DGG-2025) [ 10 ] . Specifically, we consider several coarsening methods and coefficients and evaluate our work using the strongest available explainability metrics. We also propose several novel metrics to assess quality and granularity of explanations. Importantly, to the best of our knowledge, we are the first to propose granular explainability at the assembly level for malware detection. In summary, our contributions are as follows: • We propose a novel program graph type, AFG, to represent the assembly instruction flow of a binary executable. • We capture granular explainability of AFG towards enhanced GNN transparency. • We propose a novel Meta-Coarsening approach to improve computational tractability over large graphs. • We define several novel explanation metrics to quantify explanation granularity and quality. • We consider several hyperparameters to control the size of explanations. The remainder of this paper is organized as follows. In Section II we conduct a literature review. In Section III we provide background. Section IV we cover our methodology. In Section V we

Read the complete text