Benchmarking Large Language Models for Zero-shot and Few-shot Phishing URL Detection

Najmul Hasan, Prashanth BusiReddyGari · 2026

Read Benchmarking Large Language Models for Zero-shot and Few-shot Phishing URL Detection by Najmul Hasan, Prashanth BusiReddyGari (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-02641.txt.

Description

A 2026 benchmark by Najmul Hasan and 1 co-author tests whether large language models can spot phishing URLs with little or no training examples. Deceptive links have grown far more sophisticated as criminals adopt generative AI, and phishing volume has exploded since 2022. Labeled data cannot keep pace with new tactics, which makes zero-shot and few-shot detection attractive.

The evaluation places several models under one unified prompting framework on a balanced dataset with consistent prompts. Accuracy, precision, recall, F1 score, and two ranking metrics capture both classification quality and practical utility. Operational trade-offs between models are laid out in detail. Few-shot prompting, giving the model a handful of examples, improved performance across multiple models. The authors present the benchmark as guidance for defenders choosing models for large-scale screening, where a small success rate for attackers still means many victims.

Catalog record

CREATORNajmul Hasan, Prashanth BusiReddyGari
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE21 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602-02641

Browse the full library · About the Tower of Babel