Learning to Triage Taint Flows Reported by Dynamic Program Analysis in Node.js Packages

Ronghao Ni ∗, Aidan Z.H. Yang ∗†, Min-Chien Hsu, Nuno Sabino, Limin Jia, Ruben Martins, Darion Cassel †, Kevin Cheang † · 2025

Read Learning to Triage Taint Flows Reported by Dynamic Program Analysis in Node.js Packages by Ronghao Ni ∗, Aidan Z.H. Yang ∗†, Min-Chien Hsu, Nuno Sabino, Limin Jia, Ruben Martins, Darion Cassel †, Kevin Cheang † (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-20739.txt.

Introduction

Ronghao Ni and seven co-authors test whether machine learning can sort the flood of flaw reports from program analysis tools so human reviewers see the most likely real flaws first. Their test bed is Node.js packages, where taint style flaws can let hostile input reach dangerous code.

The benchmark holds close to two thousand packages, each with one reported flaw of the studied types, drawn from a real analysis tool output. Classical models, graph networks, large language models, and hybrids are all trained and scored. The best language model leads by a small margin. At a setting that keeps ninety percent of real flaws it clears away three quarters of the harmless reports, a large saving in review time.

Catalog record

CREATORRonghao Ni ∗, Aidan Z.H. Yang ∗†, Min-Chien Hsu, Nuno Sabino, Limin Jia, Ruben Martins, Darion Cassel †, Kevin Cheang †
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE70 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2510-20739

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Learning to Triage Taint Flows Reported by Dynamic Program Analysis in Node.js Packages Ronghao Ni ∗, Aidan Z.H. Yang ∗†, Min-Chien Hsu, Nuno Sabino, Limin Jia, Ruben Martins, Darion Cassel †, Kevin Cheang † Abstract Program analysis tools often produce large volumes of candidate vulnerability reports that require costly manual review, creating a practical challenge: how can security analysts prioritize the reports most likely to be true vulnerabilities? This paper investigates whether machine learning can be applied to prioritizing vulnerabilities reported by program analysis tools. We focus on Node.js packages and collect a benchmark of 1,883 Node.js packages, each containing one reported ACE or ACI vulnerability. We evaluate a variety of machine learning approaches, including classical models, graph neural networks (GNNs), large language models (LLMs), and hybrid models that combine GNN and LLMs, trained on data based on a dynamic program analysis tool’s output. The top LLM achieves 𝑭 𝟏 = 0.915 F_{1}{=}0.915 , while the best GNN and classical ML models reaching 𝑭 𝟏 = 0.904 F_{1}{=}0.904 . At a less than 7% false-negative rate, the leading model eliminates 66.9% of benign packages from manual review, taking around 60 ms per package. If the best model is tuned to operate at a precision level of 0.8 (i.e., allowing 20% false positives amongst all warnings), our approach can detect 99.2% of exploitable taint flows while missing only 0.8%, demonstrating strong potential for real-world vulnerability triage. † † footnotetext: ∗ Equal contribution. † † footnotetext: † This work is unrelated to the authors’ employment at, or affiliations with, Amazon Web Services. † † footnotetext: Preprint. Under review. I Introduction Program analysis tools can identify potential security vulnerabilities in software, but often produce a large volume of unconfirmed vulnerability reports that require time-intensive manual examination by security analysts. Previous user studies show that developers generally prefer tools that report potential vulnerabilities with low false positives, that too many false positives is a significant pain point, and that many developers avoid tools that have too many false positives, leading to tool disuse [ 1 , 2 ] . One way to reduce false positives is to automatically synthesize proof-of-concept (PoC) exploits that demonstrate that the reported vulnerabilities can indeed be exploited by an attacker (e.g., [ 3 , 4 , 5 , 6 , 7 , 8 , 9 , 10 , 11 ] ). However, this is challenging: typically PoC exploits cannot be automatically synthesized for a large fraction of the reported vulnerabilities. Even when a PoC exploit exists, analysts still need to review whether the exploit matches the program’s threat model. Figure 1 : All triage approaches evaluated Our work aims to address this limitation of program analysis tools. We apply machine learning-based triage to prioritize the reports of existing tools by predicting which reported vulnerabilities are most likely to lead to a real-world exploit–we maintain high precision while significantly improving recall. Analysts can then focus their resources on reviewing reports with high likelihood of being true vulnerabilities. This work focuses on the Arbitrary Command Injection (ACI) and Arbitrary Code Execution (ACE) vulnerabilities [ 12 , 13 ] in Node.js JavaScript packages. The Node.js runtime is one of the most popular frameworks for web, desktop, and mobile developers. Studies have found that the Node.js ecosystem is full of packages containing vulnerabilities [ 14 , 15 , 16 , 17 ] . ACI and ACE are high-severity vulnerabilities that allow an attacker to execute code or commands on the system running the application. Previous research has introduced automated methods to identify potential ACI and ACE vulnerabilities in JavaScript programs [ 3 , 18 , 4 , 19 , 20 ] . Many of these approaches use dynamic taint analysis to detect vulnerabilities, which attempts to trace the flow of data from attacker-controlled inputs through the program to critical APIs. Dynamic analyses may report many potentially dangerous flows, but may not always indicate which ones can actually be exploited (i.e., true positives). On the other hand, static analysis techniques often generate even more false positives, further complicating the triaging process. Researchers have applied machine learning to vulnerability detection [ 21 , 22 , 23 , 24 , 25 ] , often modelling it as a classification task focusing on end-to-end detection where machine learning (ML) models are trained directly on unfiltered source code or abstract representations such as abstract syntax trees (ASTs) or control flow graphs (CFGs). These approaches typically operate on limited contexts, such as single functions or small code snippets, due to architectural constraints or dataset granularity. However, ACE and ACI vulnerability patterns can involve complex data flow dependencies which are not readily inferred from source code or textual explanations. In contrast, dynamic taint analysis tools can cover entire codebases and identify vulnerabilities across complex data flow patterns. For instance, vulnerable ACE and ACI flows can be automatically discovered by dynamic taint analysis tools like NodeMedic [ 3 ] or the more recent NodeMedic-FINE [ 4 ] , which output taint provenance graphs capturing all operations performed on attacker-controlled inputs flowing to the sink [ 3 , 4 ] . This work investigates whether work done by dynamic taint analysis tools can be directly leveraged by ML models to triage ACE and ACI vulnerabilities in Node.js packages reported by these tools. Our insights are the following: First , traditional ML models (e.g., Graph Neural Networks–GNNs) can be configured to serve as a vulnerability report triage tool by performing binary classification on the provenance graphs. Second , since recent language models pre-trained on a large corpus of code tokens have a prior understanding of code structures and general vulnerability patterns, fine-tuning these models on a dataset annotated by a dynamic taint analysis tool to include specific vulnerability patterns results in strong vulnerability triage performance. A summary of our evaluated machine learning approaches for vulnerability triage is shown in Figure 1 . LLM-only, which applies large language models directly to source code; ML, which uses classical ML models (Random Forest, XGBoost, Logistic Regression, and SVM) trained on features extracted from provenance graphs generated by the dynamic taint analysis tool NodeMedic-FINE ; GNN, which uses graph neural networks on the same provenance graphs as in ML; GNN-LLM, a hybrid model that combines embeddings from both GNNs over provenance graphs and pre-trained LLMs over source code. The baselines we compare against are program analysis and synthesis-based approaches for automatically synthesizing PoC exploits [ 3 , 4 , 5 ] . To evaluate our approaches we construct a benchmark, Triage-JS , comprising 1,883 npm packages which NodeMedic-FINE successfully detected valid potential taint flows. Our best approach, a full-fine-tuned LLM with a classification head, achieves an average F1 score of 0.915. In comparison, NodeMedic-FINE without machine learning assistance achieves only F_{1}=0.676 , failing to generate PoC exploits for a large number of vulnerabilities. For practical vulnerability triage, at less than 7% false-negative rate our best approach eliminates the need for manual review of 66.9% of non-vulnerable packages and takes around 60 ms per package . In summary, this paper makes the following contributions: • New benchmark for Node.js post-analysis triage. We introduce Triage-JS , a manually labeled benchmark of 1,883 Node.js packages, each containing one taint flow

Read the complete text