SynthChain: A Synthetic Benchmark and Forensic Analysis of Advanced and Stealthy Software Supply Chain Attacks
Zhuoran Tan, Wenbo Guo, Taylor Brierley, Jiewen Luo, Jeremy Singer, Christos Anagnostopoulos · 2026
Read SynthChain: A Synthetic Benchmark and Forensic Analysis of Advanced and Stealthy Software Supply Chain Attacks by Zhuoran Tan, Wenbo Guo, Taylor Brierley, Jiewen Luo, Jeremy Singer, Christos Anagnostopoulos (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-16694.txt.
Introduction
Tan and co-authors probe how much of supply chain attacks any telemetry can see. Modern attacks run only at runtime and scatter evidence widely. No dataset held multi-source logs with full chain ground truth. Their SynthChain set is built from real malicious packages and campaigns. Seven scenarios span PyPI, npm, and C++ chains. Annotations mark 2,919 verified compromise indicators across 22 log files. No single source recovers even 40% of attack steps. Fusing two sources lifts reconstruction about 1.6-fold.
Catalog record
| CREATOR | Zhuoran Tan, Wenbo Guo, Taylor Brierley, Jiewen Luo, Jeremy Singer, Christos Anagnostopoulos |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 106193 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.16694
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
SynthChain: A Synthetic Benchmark and Forensic Analysis of Advanced and Stealthy Software Supply Chain Attacks Zhuoran Tan, Wenbo Guo, Taylor Brierley, Jiewen Luo, Jeremy Singer, Christos Anagnostopoulos Abstract Advanced software supply chain (SSC) attacks are increasingly runtime-only and leave fragmented evidence across hosts, services, and build/dependency layers, so any single telemetry stream is inherently insufficient to reconstruct full compromise chains under realistic access and budget limits. We present SynthChain, a near-production testbed and a multi-source runtime dataset with chain-level ground truth, derived from real-world malicious packages and exploit campaigns. SynthChain covers seven representative supply-chain exploit scenarios across PyPI, npm, and a native C/C++ supply-chain case, spanning Windows and Linux, and involving four hosts and one containerized environment. Scenarios span realistic time windows from minutes to hours and are annotated with 14 MITRE ATT&CK tactics and 161 techniques (29–104 techniques per scenario). Beyond releasing the data, we quantify observability constraints by mapping each chain step to the minimum evidence needed for detection and cross-source correlation. With realistic trace availability, no single source is chain-complete: the best single source reaches only 0.391 weighted tag/step coverage and 0.403 mean chain reconstruction. Even minimal two-source fusion boosts coverage to 0.636 and reconstruction to 0.639 ( ≈ \approx 1.6× gain), with consistent chain coverage/recall improvements (0.545). The corpus contains approximately 0.58M raw multi-source events and 1.50M evaluation rows, enabling controlled studies of detection under constrained telemetry. We release the dataset, ground truth, and artifacts to support reproducible, forensic-aware runtime defenses and to guide efficient detection for software supply chains. 1 Introduction SSC compromise has become a high-leverage vector for modern adversaries and is ranked among OWASP’s top three threats in 2025 [ 42 ] . While Mandiant’s 2025 analysis attributes only 0.2% of initial intrusions directly to supply-chain compromise [ 31 ] , the downstream blast radius is often disproportionate: recent incidents show how small footholds can escalate into outsized consequences [ 28 , 1 ] . Contemporary threat intelligence further indicates that SSC campaigns are no longer isolated package-tampering events, but increasingly manifest as advanced, multi-stage, stealthy operations—including targeted manipulation of AI/ML ecosystems [ 47 ] . Echoing this shift, CrowdStrike’s 2025 threat-hunting report highlights cross-domain hands-on-keyboard activity that abuses trusted developer relationships, cloud control planes, and automation [ 7 ] . These trends expose a fundamental gap: advanced supply-chain attacks unfold across multiple stages and extend beyond source code alone , and the evidence needed to detect and reconstruct them is fragmented across heterogeneous telemetry sources, much of it generated at runtime . Yet existing research and benchmarks predominantly emphasize static artifact inspection (e.g., malicious package discovery) [ 11 , 16 ] or dynamic analysis in limited sandbox settings [ 41 , 55 ] . As a result, available datasets offer only partial visibility: static corpora lack execution semantics and post-compromise workflows, while many dynamic datasets rely on relatively simple triggers and do not model the multi-stage, cross-environment distributed behaviors typical of modern supply-chain intrusions. Consequently, practitioners and researchers are often forced to study supply-chain compromise either without realistic runtime traces or without the multi-source evidence required for chain reconstruction and forensic validation. Key challenge: observability limits in realistic deployments. In real environments, defenders seldom have “full fidelity” visibility: telemetry is constrained by access boundaries (e.g., managed services, proprietary build systems), cost and performance budgets, and operational trade-offs (sampling, retention, and source coverage) [ 9 ] . For advanced supply-chain intrusions, these constraints are not incidental—they directly determine what is detectable. Evidence is frequently non-redundant across sources: a chain step visible in process lineage can be absent from system logs; a network indicator can be inconclusive without service traces; and pipeline-side artifacts can be inaccessible at runtime [ 18 ] . This implies that single-source detection is inherently incomplete for advanced supply-chain attacks: even an ideal detector operating on a single stream cannot recover a complete compromise chain when required evidence is missing by design [ 45 ] . Our approach. Guided by techniques observed in large-scale malicious open-source software (OSS) packages [ 11 ] , we select representative real-world samples to build a near-production supply-chain attack testbed and a multi-source runtime dataset. We collect synchronized telemetry across hosts, services, and containers—including process lineage, system/audit logs, and network/service traces, with container-level visibility enabled via eBPF-based instrumentation—to support chain-level analysis. We construct chain-level ground truth by combining (i) technique-level adversary actions directly exported from Mythic 1 1 1 An open-source C2 framework that records operator tasking and implant responses, enabling direct export of executed actions and their ATT&CK technique mappings. C2 tasking logs (ATT&CK-mapped by construction for C2-driven steps) with (ii) payload-originated actions extracted via an LLM-assisted pipeline with manual verification. We then align defender-visible events to these actions using coarse rule/keyword matching to enable chain reconstruction and detectability analysis (Section 4 ). Finally, we evaluate the marginal benefit of each telemetry stream under realistic availability constraints by comparing detection and chain-reconstruction performance for single-source, two-source, and multi-source fusion. This characterizes observability limits and informs cost-effective telemetry choices for runtime SSC defense. This design balances realism with experimental control, and isolates observability (telemetry anchors and cross-source joins) as the primary limiting factor—rather than confounding results with complex alignment or detector-specific assumptions. What we find: single-source is not enough while more is not always better. Our key finding is twofold: single-source monitoring is inherently insufficient for chain-complete evidence, while effective detection depends on which sources are fused rather than simply adding more. Our evaluation shows that no single source provides chain-complete evidence: the best single source reaches only 0.391 weighted tag/step coverage and 0.403 mean chain reconstruction. In contrast, even a minimal two-source fusion raises coverage to 0.636 and reconstruction to 0.639 (approximately 1.6 \times gain), with consistent improvements in chain coverage/recall (0.545). Importantly, the gains are not monotonic with the number of sources: additional streams can introduce noise and redundant evidence, yielding diminishing returns unless correlation is targeted. Overall, multi-source correlation is a prerequisite for reconstructing end-to-end supply-chain compromise chains under realistic observability limits , and cost-effective telemetry requires selecting complementary sources rather than maximizing collection . To summarize, the contributions include: • End-to-end supply-chain scenarios grounded in real incidents. We distill exploitation prerequisites and runtime compromise patterns from recent incidents and instantiate them into seven representative end-to-end scenarios spanning multiple ecosystems and platforms. • A near-production testbed and multi-source runtime dataset with chain-level ground truth. We collect synchronized host-, process-, network-, and service-level telemetry and align it to ground-truth action timelines to enable chain-level analysis. • ATT&CK-aligned annotations and exploitation-trend analysis. We provide ATT&CK-mapped labels and analyses that expose recurring compromise structures, exploitation trends, and cross-scenario indicators. • Chain-level detectability and telemetry trade-off study. We quantify chain-level detectability under telemetry constraints, showing that single-source evidence is inherently incomplete and that chosen multi-source fusion substantially improves reconstruction. To the best of our knowledge, SynthChain is the first public dataset that jointly captures: (i) end-to-end execution traces grounded in real supply-chain compromise paths; (ii) multi-stage post-compromise activity; and (iii) synchronized multi-source telemetry enabling chain-level reconstruction and evaluation under realistic observability constraints. 2 Limitations of Existing Telemetry for Advanced Supply-Chain Attack Analysis Advanced SSC compromises are normally mediated by package ecosystems, dependency resolution, and build/release pipelines,