AgentGuard: A Multi-Agent Framework for Robust Package Confusion Detection via Hybrid Search and Metadata-Content Fusion

Yu Li, Wei Ma, Zhi Chen, Ye Liu, Lingxiao Jiang, Junyi Tao, Hao Liu, Yongqiang Lyu, Qiang Hu ✉ · 2026

Read AgentGuard: A Multi-Agent Framework for Robust Package Confusion Detection via Hybrid Search and Metadata-Content Fusion by Yu Li, Wei Ma, Zhi Chen, Ye Liu, Lingxiao Jiang, Junyi Tao, Hao Liu, Yongqiang Lyu, Qiang Hu ✉ (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-16309.txt.

Introduction

Li and eight colleagues present AgentGuard, a multi-agent framework for detecting package confusion in open-source software supply chains. Attackers publish harmful packages with names close to legitimate ones, meant to mislead. Older detectors rely on a single signal, suffer high false positives, and face evasion tricks. AgentGuard first finds confusion targets with fine-tuned word models and hybrid similarity matching. It then fuses metadata and content signals to separate benign look-alikes from harmful fakes with different code.

Catalog record

CREATORYu Li, Wei Ma, Zhi Chen, Ye Liu, Lingxiao Jiang, Junyi Tao, Hao Liu, Yongqiang Lyu, Qiang Hu ✉
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE59924
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.16309

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

AgentGuard: A Multi-Agent Framework for Robust Package Confusion Detection via Hybrid Search and Metadata-Content Fusion Yu Li, Wei Ma, Zhi Chen, Ye Liu, Lingxiao Jiang, Junyi Tao, Hao Liu, Yongqiang Lyu, Qiang Hu ✉ Abstract The proliferation of open-source software (OSS) has made software supply chains prime targets for attacks like Package Confusion, where adversaries publish malicious packages with names deceptively similar to legitimate ones. To protect against such attacks and safeguard the use of OSS, multiple confusion detection methods have been proposed. However, existing methods are limited to single-signal retrieval strategies (relying solely on lexical or semantic metrics), struggle with high false positive rates (FPR), and are vulnerable to adversarial evasion. Critically, as content-agnostic approaches, they fundamentally fail to distinguish benign packages with high naming similarity from malicious, code-dissimilar impersonations, leading to persistent high FPR. To address these limitations, we introduce AgentGuard, a novel multi-agents based framework for package confusion detection. Specifically, it first discovers potential confusion targets using fine-tuned word embedding models with hybrid similarity search. After that, It subsequently evaluates risk via a fused machine learning model that uniquely combines: (1) a multi-dimensional metadata group and (2) a novel package content analysis group, to reduce the FPR and mitigate the impact of adversarial evasion. To assess the effectiveness of AgentGuard, we evaluate it on challenging ConfuDB and NeupaneDB datasets. Our results demonstrate that AgentGuard significantly outperforms state-of-the-art baselines, ConfuGuard and Typomind, improving precision by 12%-49% while simultaneously reducing the FPR by 11%-35%, and effectively discovers the confused package. ✉ ✉ footnotetext: Corresponding author: [email protected] I Introduction Modern software development relies heavily on open-source package ecosystems, with registries like NPM npm, Inc. (2025) and PyPI Python Software Foundation (2024) hosting millions of packages and serving billions of weekly downloads NPM Contributors (2024) ; Soto-Valero et al. (2019) ; Schorlemmer et al. (2024) . This scale and openness create significant attack surfaces for software supply chain threats Scalco et al. (2022) ; Vu et al. (2021) ; Herr (2020) ; Vu et al. (2020a) ; SonaType (2021) . Among these, package confusion attacks pose a particularly insidious risk: adversaries publish malicious packages with names designed to closely resemble legitimate ones, deceiving developers into installing them and achieving arbitrary code execution. These attacks exploit diverse confusion mechanisms spanning lexical (typosquatting like bz2file and bz2fiel ), syntactic (delimiter modifications, reordering or scope default like bz2file and @xx/bz2file ), and semantic (synonym substitution like bz2file and bzip ) levels Vu et al. (2020b) ; Neupane et al. (2023) . To defend against these attacks and ensure the safe usage of OSS, multiple package confusion detection methods have been proposed. However, these methods have evolved from simple lexical matching to semantic analysis and metadata-based filtering, and therefore, are still facing critical limitations: Ineffective Target Discovery Existing methods typically rely on a single signal, such as purely semantic or purely lexical search, and thus, fail to capture the diversity of confusion attacks. For example, semantic-only search misses obvious lexical typos, while syntactic-only search is blind to semantic attacks, resulting in poor target recall. High False-Positive Rates Naming similarity is widespread. Crucially, existing content-agnostic detectors fundamentally struggle to distinguish malicious impersonation from legitimate packages that incidentally similar names and metadata, leading to excessive false positives and alert fatigue. Susceptibility to Adversarial Evasion Defenses relying on static rules or easily manipulable metadata features remain fragile against attackers who can forge or obfuscate these signals. To address these challenges, we introduce AgentGuard, a multi-agent framework that integrates a novel, fused classification model for package confusion detection. First, addressing the ineffective target discovery challenge, AgentGuard employs a hybrid (semantic and syntactic) search strategy. This strategy combines fine-tuned semantic embeddings for conceptual recall with syntactic trigram similarity for lexical precision, enabling it to discover confusion targets effectively. Second, focusing on the limitations of high FPR and adversarial evasion, we design a fused machine learning model that moves beyond metadata-only analysis. This classifier is fortified with two distinct, complementary feature sets: (1) a multi-dimensional metadata feature group, which incorporates robust temporal signals (TS) to resist forgery, and (2) our proposed package content (PC) feature group. The PC features provide a compelling signal by comparing package size differences, file list similarity, dependency graphs, and code embeddings to distinguish benign packages with malicious mimics, simultaneously boosting recall and precision. To assess the effectiveness of AgentGuard, we conduct comprehensive experiments on two commonly used datasets, ConfuDB and NeupaneDB. The results demonstrated that, compared to the SOTA methods ConfuGuard and Typomind, AgentGuard achieved up to 49% higher precision scores with 35% lower FPR. Our ablation studies showcase the importance of our agent design and feature selection. To summarize, the main contributions of this paper are: • We propose a hybrid search strategy that integrates semantic and syntactic signals to effectively discover confusion targets, outperforming single similarity metric approaches. • We introduce a novel package content features set (comprising 4 features: packages size ratio, file list similarity, dependency similarity, and code embeddings similarity) that quantifies code-level plausibility to systematically reduce false positives and accurately identify malicious impersonations. • We design a robust detection model grounded in multi-dimensional feature analysis. By combining the proposed package content signals with temporal and metadata quality features, our model establishes a resilient verification mechanism that effectively withstands adversarial metadata manipulation and forgery. • We design AgentGuard, the first multi-agent based package confusion detection framework with SOTA performance and broad ecosystem adaptability. This article extends our preliminary work Ma et al. (2025) . Compared to the conference version, we introduce two novel modules to enhance AgentGuard: First, to address the limitation of ineffective target discovery, we implement a hybrid search strategy that autonomously and accurately identifies the correct legitimate target. Second, to fundamentally resolve the High FPR challenge, we introduce a deep package content analysis module. This module performs content-aware verification by comparing package structure (e.g., size, file lists, dependencies) and code-based vector similarity against the discovered target. This critical enhancement enables the model to precisely distinguish malicious mimics from benign packages with high naming similarity, ensuring robust detection without excessive false alarms. Finally, we extensively evaluate these modules through controlled ablation studies, demonstrating that our fused approach achieves both superior baseline performance and best-in-class adversarial robustness. II Background and Related Work II-A Software Supply Chain Security Software supply chain attacks have become a central challenge in modern cybersecurity Scalco et al. (2022) ; Herr (2020) ; SonaType (2021) ; Kalu et al. (2025) . Unlike traditional attacks, these methods contaminate third-party components, such as open-source packages, relied upon during development Amusuo et al. (2025) . The event-stream incident is a canonical example of how injecting malicious code into a popular dependency can propagate threats to thousands of downstream projects. Package confusion is a specific form of supply chain attack that targets the human developer through social engineering Kaplan and Qian (2021) . The concept of namesquatting is well-studied in other domains, such as DNS and mobile application markets Moubayed et al. (2018) . Package confusion adapts this idea to software package registries, where adversaries exploit naming similarities to deceive developers into installing malicious code Soto-Valero et al. (2019) ; Schorlemmer et al. (2024) . Consistent with established taxonomy Vu et

Read the complete text