HackerSignal : A Large-Scale Multi-Source Dataset Linking Hacker Community Discourse to the CVE Vulnerability Lifecycle

Benjamin M. Ampel, Sagar Samtani · 2026

Read HackerSignal : A Large-Scale Multi-Source Dataset Linking Hacker Community Discourse to the CVE Vulnerability Lifecycle by Benjamin M. Ampel, Sagar Samtani (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-03158.txt.

Introduction

Benjamin M. Ampel and Sagar Samtani release HackerSignal, a large dataset linking hacker forum talk to the full life cycle of software vulnerabilities. Threat analysts need to connect early chatter, exploits, advisories, and fixes, but those records usually sit in separate silos.

The set holds millions of deduplicated documents from dozens of sources across eight layers and more than three decades, linked by shared CVE identifiers. Three benchmark tasks are shown: cross-source linkage, exploit type classification, and forward-in-time generalization with disjoint vulnerabilities. Leakage checks, audit packets, and governance notes ship with the release for careful use.

Catalog record

CREATORBenjamin M. Ampel, Sagar Samtani
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE41 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.03158

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

HackerSignal : A Large-Scale Multi-Source Dataset Linking Hacker Community Discourse to the CVE Vulnerability Lifecycle Benjamin M. Ampel, Sagar Samtani Abstract We introduce HackerSignal, a benchmark for temporal out-of-distribution cyber threat intelligence (CTI) and cross-source CVE linkage. HackerSignal aggregates 7.45 million exact-deduplicated documents from 64 public forum/source identifiers spanning eight source layers and a 36-year window (1990–2026). In contrast to other publicly accessible cybersecurity datasets, HackerSignal is among the first public benchmark datasets that maps the full potential exploit to vulnerability trajectory from hacker community discourse, exploit databases with working and proof of concept exploits, vulnerability advisories, and software fix commits. HackerSignal creates these linkages through a shared CVE identifier space while preserving source-specific release modes to support a range of unique Artificial Intelligence (AI)-enabled cybersecurity analytics tasks. In this paper, we summarize HackerSignal and illustrate three selected benchmark tasks it uniquely supports: (1) CVE linkage retrieval (cross-source temporal out-of-distribution entity grounding); (2) exploit type classification (8-class vulnerability type prediction with temporal OOD evaluation); and (3) temporal generalization (prospective CVE-disjoint evaluation where C train ∩ C test = ∅ C_{\text{train}}\cap C_{\text{test}}=\emptyset ). All tasks use temporal splits to evaluate prospective generalization. We release source-shortcut and leakage diagnostics, manual-audit packets, a datasheet, and a release-governance addendum to support the dissemination of the dataset. HackerSignal’s code, data, and Croissant metadata are available at hf.co/datasets/BenAmpel/HackerSignal (data) and github.com/BenAmpel/hackersignal (code). 1 Introduction Understanding how cyber threats emerge, propagate, and relate to vulnerabilities is a central challenge in modern applied cybersecurity. Signals that tracks this trajectory is distributed across heterogeneous text-based sources, including: (1) hacker forums where exploits are first discussed, (2) public exploit databases where proof-of-concept code is released, (3) official vulnerability advisories such as the National Vulnerability Database, CISA’s Known Exploited Vulnerabilities, and GitHub Advisory where threats are cataloged, and (4) open-source repositories where fixes are committed. Extant public datasets in this domain often span a single or subset of these layers ( Pastrana et al. 2018 ) and are usually not publicly released ( Ampel et al. 2020 ) . A survey of CTI extraction from text identifies advisory records and threat reports as the dominant labeled sources studied, with exploit-archive and PoC-focused benchmarks remaining scarce ( Rahman et al. 2023 ) ; a companion survey of cybercrime community research corroborates that structured datasets linking forum posts to formal vulnerability records are largely absent from the public literature ( Hughes et al. 2024 ) . HackerSignal addresses this gap with a temporally structured, multi-source corpus and benchmark spanning the full advisory-to-exploit trajectory, enabling downstream CTI analytics tasks. HackerSignal makes three key contributions: 1. A temporally grounded CTI corpus of 7.45M documents from 64 public forum/source identifiers across eight source layers (hacker communities, exploit archives, vulnerability references, exploit Q&A references, advisories, bug-bounty disclosures, fix commits, and exploitation references), with CVE linkage metadata connecting vulnerability identifiers to the texts that discuss them. The 6.97M-row hacker community subcorpus alone (spanning 56 forums across darkweb marketplaces, security discussion boards, and multilingual hacker communities) supports diverse NLP tasks beyond our three benchmarks, including exploit code summarization, security content classification, topic modeling, community interaction analysis, and cross-lingual CTI extraction. 2. Three benchmark tasks with temporal train/val/test splits that test progressively harder generalization in (1) cross-source CVE linkage retrieval, (2) 8-class exploit type classification, and (3) CVE-disjoint temporal generalization ( C_{\text{train}}\cap C_{\text{test}}=\emptyset ), accompanied by shortcut/leakage diagnostics. 3. A full model ladder and governance artifacts from BM25 through four dense bi-encoders (MiniLM, mpnet, BGE, E5), bag-of-words classifiers (Decision Tree, TF-IDF+LR, SVM), recurrent neural networks (RNN, GRU, LSTM, BiLSTM), a domain-specific transformer (SecBERT), and hybrid sparse+dense retrieval, alongside source-shortcut diagnostics, manual-audit packets, and a release-governance addendum. Figure 1 provides an overview of the complete pipeline of the proposed dataset, including eight source layers unified through a shared CVE identifier space, then channeled into three benchmark tasks with temporal controls. Temporal splits are necessary in our context as real CTI systems must generalize to threats that did not exist at training time ( Koh et al. 2021 ) . We explicitly audit the dataset for temporal and source leakage ( Kapoor and Narayanan 2023 ) because benchmark claims are otherwise easy to inflate (discussed in Section 3. 2 Related Work Hacker-community cyber threat intelligence. Early hacker-forum CTI work treated underground communities as a source of proactive intelligence about tools, actors, and emerging attack capabilities ( Samtani et al. 2015 ; Samtani et al. 2017 ) . The AZSecure Hacker Assets Portal operationalized this as a situational awareness platform ( Samtani et al. 2021 ) , while parallel work modeled threat emergence over time via diachronic graph embeddings ( Samtani et al. 2020 ) and mobile malware identification ( Grisham et al. 2017 ) . The resulting artifacts are operational systems rather than reusable ML benchmarks, since their data remains proprietary. Rahman et al. 2023 survey automated CTI extraction from text, observing that collections often concentrate on the advisory and threat-report tier, while exploit-archive and PoC-focused benchmarks remain scarce. Hughes et al. 2024 systematize research on the cybercrime community and demonstrate a persistent shortage of ground-truth datasets. Most publicly collected forum data lacks structured labels linking posts to formal vulnerability records. Exploit labeling and vulnerability linkage. The closest predecessor to HackerSignal is in the field of exploit labeling research. Prior methods labeled hacker-forum exploit source code via deep transfer learning from professional exploit repositories ( Ampel et al. 2020 ) and later extended this with pre-initialization, multi-layer transfer, and self-attention ( Ampel et al. 2024 ) . For cross-source linkage, an attention-based deep-structured semantic model linked dark-web exploits to known vulnerabilities, yielding 52,590 linkages ( Samtani et al. 2022 ) . HackerSignal differs by releasing a governed, temporally split public benchmark that exposes source provenance, weak-label caveats, and a quality-controlled CVE-linkage retrieval task. Finally, the hacker forum collection of Otto et al. 2021 aggregates 259K posts from three English-language security communities (AntiOnline, Go4Expert, CipherSpace) and studies exploit-sharing behavior via graph embeddings. Underground community datasets. Research has also focused on the general crawling of hacker communities, like a credentialized dataset of darkweb cybercrime forum posts widely used for crime and exploit prediction ( Pastrana et al. 2018 ) Nunes et al. 2016 presented an operational darknet/deepnet mining system over hacker forums and markets for proactive CTI. The Gayanku ( Gayanku 2020 ) dataset contains 3.8M posts from seven darknet marketplaces (Silk Road, Agora, Evolution, etc.) on Kaggle. Exploit and vulnerability databases. ExploitDB ( Offensive Security 2003–2026 ) is the salient public exploit archive (50K+ entries, many with CVE IDs). The CVEfixes dataset ( Bhandari et al. 2021 ) links CVEs to fix commits in open-source repositories. NVD ( NIST 2002–2026 ) , CISA KEV ( CISA 2021–2026 ) , and GitHub Advisory ( GitHub 2017–2026 ) are official advisory sources. HackerSignal normalizes all five into a common JSONL schema with CVE keys. Multilingual CTI. More general hacker-forum text classification work compared SVM and CNN models for extracting CTI from forum posts ( Deliu et al. 2017 ) , while Ebrahimi et al. 2020 showed that non-English hacker forums require cross-lingual approaches that avoid brittle machine translation. CVE severity prediction ( Han et al. 2017 ) , exploit labeling ( Ampel et al. 2024

Read the complete text