PROVEX: Enhancing SOC Analyst Trust with Explainable Provenance-Based IDS

Devang Dhanuka, Prof. Nidhi Rastogi · 2025

Read PROVEX: Enhancing SOC Analyst Trust with Explainable Provenance-Based IDS by Devang Dhanuka, Prof. Nidhi Rastogi (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2512-18199.txt.

Introduction

Devang Dhanuka and Nidhi Rastogi at the Rochester Institute of Technology build PROVEX, an explainability layer for intrusion detection in security operations centers. Modern detectors use graph neural networks on system provenance data, but their alerts arrive as black box verdicts that analysts struggle to trust. PROVEX sits on top of KAIROS, a leading temporal graph based detector. Its design can adapt to other temporal graph detectors with little change.

The framework adds explanations after detection, showing why an alert fired by naming the key causal subgraphs and events. Three explanation methods, GraphMask, GNNExplainer, and a temporal variant, are adapted to provenance graphs. On the DARPA CADETS Engagement 3 dataset, the explainers preserve the detector's decisions with high fidelity. They surface telling edges such as malicious file interactions and anomalous network flows. Each explanation costs only 3 to 5 seconds per event. The goal is faster triage and justified trust in automated alerts.

Catalog record

CREATORDevang Dhanuka, Prof. Nidhi Rastogi
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE54 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2512-18199

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

PROVEX: Enhancing SOC Analyst Trust with Explainable Provenance-Based IDS Devang Dhanuka, Prof. Nidhi Rastogi Abstract Modern intrusion detection systems (IDS) leverage graph neural networks (GNNs) to detect malicious activity in system provenance data, but their decisions often remain a “black box” to analysts. This paper presents a comprehensive XAI framework designed to bridge the trust gap in Security Operations Centers (SOCs) by making graph-based detection transparent. We implement this framework on top of KAIROS, a state-of-the-art temporal graph-based IDS, though our design is applicable to any temporal graph-based detector with minimal adaptation. The complete codebase is available at https://github.com/devang1304/provex.git . We augment the detection pipeline with post-hoc explanations that highlight why an alert was triggered, identifying key causal subgraphs and events. We adapt three GNN explanation methods – GraphMask, GNNExplainer, and a variational temporal GNN explainer (VA-TGExplainer) – to the temporal provenance context. These tools output human-interpretable representations of anomalous behavior, including important edges and uncertainty estimates. Our contributions focus on the practical integration of these explainers, addressing challenges in memory management and reproducibility. We demonstrate our framework on the DARPA CADETS Engagement 3 dataset and show that it produces concise window-level explanations for detected attacks. Our evaluation reveals that the explainers preserve the TGNN’s decisions with high fidelity, surfacing critical edges such as malicious file interactions and anomalous netflows. The average explanation overhead is 3–5 seconds per event. By providing insight into the model’s reasoning, our framework aims to improve analyst trust and triage speed. I Introduction Machine-learning driven IDSs have shown great success in detecting advanced threats in complex system logs, but they often lack transparency. In security operations, this opacity creates a critical “trust gap”: a detector that raises an alert without explanation provides little insight into whether the alert is a true attack or a false positive. KAIROS is a state-of-the-art provenance-based IDS that uses a Temporal Graph Neural Network (TGNN) to detect anomalies in streaming system event graphs with high accuracy [ 5 ] . However, its decision-making process remains a black box. When KAIROS flags a sequence of events as malicious, analysts are presented with a reconstructed attack subgraph but lack specific guidance on why the model considered those events anomalous. To bridge this gap, we present a comprehensive XAI framework implemented as PROVEX, which augments the KAIROS pipeline with post-hoc explanations. Our goal is not to propose a novel explanation algorithm, but to adapt existing GNN explainability techniques to the specific constraints of provenance-based intrusion detection. We focus on generating temporal subgraph explanations that highlight the key causal events driving the model’s alerts. A core contribution of our work is the integration of a Variational Temporal Graph Explainer (VA-TGExplainer). While standard methods like GNNExplainer [ 6 ] and GraphMask [ 7 ] provide valuable insights, they often struggle with the ambiguity inherent in provenance graphs, where multiple plausible explanations may exist for a single anomaly. Furthermore, deterministic masks can be unstable. VA-TGExplainer addresses this by modeling the explanation as a distribution, capturing the uncertainty in edge importance. This allows us to provide analysts with not just a single explanation, but a measure of confidence in which edges are truly critical versus those that are merely incidental. Our pipeline combines global window-level pruning (via GraphMask) with fine-grained, uncertainty-aware event analysis (via VA-TGExplainer) to provide a comprehensive view of the detected threat. In this paper, we detail how our framework addresses these challenges and achieves explainable intrusion detection on the CADETS dataset. We make the following contributions: • Explainable TGNN Framework: We design an end-to-end pipeline that combines KAIROS’s TGNN-based detection with multiple GNN explainers to produce human-readable explanations for each detected attack. To our knowledge, this is the first deployable system providing temporally-aware GNN explanations for streaming cyber attack detection. • Adaptation of XAI Methods: Instead of proposing a new explainer algorithm, we adapt and engineer existing methods for our domain. We integrate GraphMask for identifying critical edges in an entire attack window, GNNExplainer [ 6 ] for fine-grained per-event explanation with fidelity metrics, and a custom VA-TGExplainer (a variational temporal graph explainer) that extends prior temporal explainers to quantify uncertainty in the importance of each edge. These methods are modified to work on KAIROS’s temporal graph data and are orchestrated to provide both global (window-level) and local (event-level) insights. • Scalable Framework Design: We implement a suite of system improvements to handle the scale and complexity of real audit logs. This includes GPU memory management with caching and automatic fallback to CPU when GPU memory is insufficient, parallel processing of time windows for throughput, and logging/reproducibility features to ensure consistent results despite nondeterministic components. Our caching of intermediate graph data cuts peak memory usage by \approx 60% in tests, avoiding out-of-memory errors when explaining large subgraphs. Fallback logic ensures the pipeline doesn’t fail on resource-constrained systems. • Evaluation: We evaluate the framework on the DARPA Transparent Computing (TC) Engagement 3 dataset [ 5 ] (April 2018), focusing on the CADETS host’s attack on April 6, 2018. The system successfully generates explanations for the known attack behaviors (e.g., a webserver spawning a shell and exfiltrating data) by highlighting the most anomalous edges. We report metrics on explanation fidelity and computational cost, demonstrating the feasibility of deploying this in a SOC setting. II Related Work Effective intrusion detection and explainability in dynamic graphs are active areas of research. This section reviews key developments in provenance-based IDS and the evolution of GNN explainability methods. II-A Intrusion Detection on Provenance Graphs Provenance-based intrusion detection has evolved from rule-based systems to advanced learning-based approaches. Early systems like StreamSpot [ 10 ] used graph clustering on streaming edges to identify anomalies, while Unicorn [ 4 ] employed graph sketching to summarize provenance graphs for anomaly detection. HOLMES [ 3 ] represented a significant step in correlating information flows to detect APTs, though it relied on expert-defined rules. More recently, KAIROS [ 5 ] (IEEE S&P 2024) advanced the state-of-the-art by utilizing Temporal Graph Neural Networks (TGNNs) [ 2 ] to learn dynamic node embeddings from streaming audit logs, enabling the detection of unseen attacks without manual rules. Other deep learning approaches like DeepLog [ 16 ] focus on system logs but lack the structural richness of provenance graphs. Provenance capture mechanisms like CamFlow [ 14 ] and SPADE [ 15 ] provide the foundational data for these systems. However, while KAIROS provides an “attack reconstruction” subgraph, it lacks an intrinsic mechanism to explain why a specific event sequence triggered an alert, a gap our work addresses. II-B Static GNN Explainability Explainability for static Graph Neural Networks is well-established, building on general ML interpretability methods like LIME [ 12 ] and SHAP [ 13 ] . GNNExplainer [ 6 ] is a seminal perturbation-based method that learns a soft mask to maximize mutual information between the masked graph and the prediction. GraphMask [ 7 ] improves upon this by learning a mask via a differentiable objective to find a minimal sufficient subgraph. PGExplainer [ 11 ] trains a parameterized explainer network to generate masks inductively, offering faster inference than optimization-based methods. Other approaches like SubgraphX [ 8 ] use Monte Carlo Tree Search to explore subgraphs, while XGNN [ 17

Read the complete text