PhishSense-1B: A Technical Perspective on an AI-Powered Phishing Detection Model

S.E. Blake · 2025

Read PhishSense-1B: A Technical Perspective on an AI-Powered Phishing Detection Model by S.E. Blake (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-10944.txt.

Introduction

Blake presents a 2025 technical report on spotting phishing emails with a language model. Phishing mail tricks readers into giving away passwords or money, and it stays a steady threat. The report adapts a small guard model, Llama-Guard-3-1B, into a tool tuned just for this task.

The tuning used a low-rank method and a balanced set of phishing and harmless mail. The tuned model, named PhishSense-1B, reached very high accuracy on the team's own test set. On a harder real-world set its score was lower but still beat the base model and older BERT-based tools. The report also weighs prompt tricks against fine-tuning and sketches where the tool could be deployed.

Catalog record

CREATORS.E. Blake
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE24 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2503-10944

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

PhishSense-1B: A Technical Perspective on an AI-Powered Phishing Detection Model S.E. Blake Abstract Phishing remains one of the most persistent cybersecurity threats in the digital era. In this paper, we present Phishsense-1B —a fine-tuned variant of the meta-llama/Llama-Guard-3-1B model adapted for phishing detection and reasoning via Low-Rank Adaptation (LoRA) and the GuardReasoner finetuning methodology Liu et al. 2025 . We detail our LoRA-based fine-tuning methodology, describe the balanced dataset of phishing and benign emails, and demonstrate dramatic performance gains over the base model. Our experiments show that Phishsense-1B achieves near-perfect recall with an accuracy of 97.5% on a custom dataset and maintains robust performance (70% accuracy) on a challenging real-world dataset jov , significantly outperforming both unadapted and BERT-based detectors. Additionally, we review current state-of-the-art detection methods, compare prompt-engineering with fine-tuning approaches, and discuss potential deployment scenarios. 1 Introduction Phishing attacks continue to impose a significant threat on digital communication and online transactions, costing organizations and individuals billions of dollars each year. According to the Anti-Phishing Working Group (APWG), phishing incidents increased by over 25% in 2022 compared to previous years, with attackers refining their methods to mimic trusted brands and deceive users into revealing sensitive information Anti-Phishing Working Group 2022 . This alarming increase not only highlights the ingenuity of cybercriminals but also emphasizes the critical need for more advanced detection systems. In response, researchers and cybersecurity professionals have increasingly turned to artificial intelligence (AI) and deep learning (DL) techniques to build more accurate and adaptable detection systems capable of identifying subtle cues in phishing attempts. Historically, phishing detection relied on signature-based methods and blacklists, which, although useful, could not keep pace with the rapid evolution of phishing tactics. Traditional approaches often suffered from high false-positive rates and were unable to adapt to new, previously unseen attack vectors. In contrast, the advent of deep learning has allowed for the development of models that can automatically learn relevant features from raw data, reducing the need for manual feature engineering. Recent studies employing deep learning methods have reported striking performance improvements. For instance, long short-term memory (LSTM)-based models have achieved accuracies as high as 99.1% on phishing email datasets Yang et al. 2024 , demonstrating their capability to capture temporal dependencies and subtle patterns in textual data. In parallel, researchers have explored convolutional neural networks (CNNs) for detecting phishing URLs by focusing on character-level information. Character-level CNN architectures have reached detection rates of up to 98.74% for URL-based phishing detection Shweta et al. 2021 . These models are particularly effective because they do not rely on pre-defined features but instead learn to extract discriminative patterns directly from the input strings. Hybrid approaches that combine CNNs with LSTMs have also been developed, leveraging the spatial feature extraction capabilities of CNNs along with the temporal sequence learning strengths of LSTMs Quang et al. 2020 , Adebowale et al. 2020 . Such combinations can capture both local patterns (e.g., specific character sequences or sub-strings common in phishing URLs) and the overall sequence structure, thereby providing a more robust detection mechanism. Despite these promising results, deploying state-of-the-art deep learning models in real-world settings poses several challenges. One major challenge is the computational cost associated with fine-tuning large language models (LLMs). Traditional fine-tuning methods typically require updating millions of parameters, leading to high memory usage and long training times. To address these issues, researchers have recently introduced parameter-efficient fine-tuning techniques such as Low-Rank Adaptation (LoRA). LoRA updates only a small subset of parameters—sometimes as little as 0.3% to 0.6% of the original model weights—while keeping the bulk of the pre-trained model frozen Meo et al. 2024 . This approach significantly reduces computational overhead and memory requirements, making it feasible to deploy sophisticated phishing detection systems even on resource-constrained devices. For example, a recent study fine-tuned a BERT-based phishing detector using LoRA and maintained an accuracy of 98% on a dataset comprising over 650,000 URLs, while significantly reducing both training time and memory consumption Aslam et al. 2024 . The success of this parameter-efficient strategy underscores the potential for deploying robust phishing detection systems in real-world environments where computational resources may be limited. The practical implications of these advancements are clear. By combining robust deep learning architectures with efficient fine-tuning strategies such as LoRA, it is now possible to deploy real-time phishing detection systems that operate effectively across a wide range of platforms, from enterprise servers to mobile devices. In our work, we investigate three DL architectures—LSTM, CNN, and a hybrid LSTM–CNN model—evaluated on a dataset of 20,000 URLs with 80 extracted features (which were reduced to 30 via feature selection). Our experimental results show that the CNN model achieves an accuracy of 99.2%, outperforming both the LSTM (96.8%) and the hybrid model (97.6%). These findings highlight the significant role that model architecture and parameter tuning play in achieving optimal performance in phishing detection tasks. Beyond performance metrics, our study emphasizes several critical aspects of modern phishing detection. First, the dynamic and evolving nature of phishing attacks requires detection systems that are not only accurate but also adaptable. Phishers continually alter their tactics to bypass security systems, which means that detection models must be regularly updated and fine-tuned. Techniques like LoRA facilitate rapid domain adaptation, allowing models to quickly incorporate new patterns without the need for exhaustive retraining. Second, the integration of multiple data sources is becoming increasingly important. Traditional detection methods often relied solely on email content or URL strings; however, modern approaches incorporate additional contextual data such as website metadata, third-party threat intelligence, and user behavior analytics. By leveraging multiple data sources, deep learning models can achieve a more comprehensive understanding of phishing attempts, further reducing false positives and negatives. Third, the trade-off between model complexity and deployment feasibility remains a key challenge. While deep learning models have achieved impressive accuracy rates in controlled experimental settings, their real-world deployment requires careful consideration of computational constraints, latency, and scalability. Parameter-efficient methods like LoRA help bridge this gap by enabling the deployment of complex models in environments with limited resources, such as mobile devices or edge computing platforms. Finally, our work contributes to a growing body of literature that aims to provide a concrete analysis of state-of-the-art DL-based phishing detection. We discuss not only the performance achievements but also the challenges that remain in this field. These challenges include dealing with imbalanced datasets, managing false positives in operational environments, and ensuring that models can adapt to rapidly changing attack vectors without compromising detection accuracy. In conclusion, the evolution of phishing detection has been marked by significant advancements driven by deep learning and parameter-efficient fine-tuning techniques. The combination of CNNs, LSTMs, and hybrid architectures with methods like LoRA represents a promising direction for future research and practical application. By grounding our approach in empirical results and leveraging advanced techniques, we contribute a practical framework that can be directly applied in real-world cybersecurity scenarios. Our research not only demonstrates high detection accuracy but also provides insights into how these advanced methods can be further refined to meet the ever-changing demands of digital security. 2 Related Work Phishing detection has been an active area of research for many years, with early approaches primarily relying on traditional machine learning techniques. These

Read the complete text