AI-Powered Image Analysis for Phishing Detection PubID: pubid: This is a preprint version of the paper accepted for the 9th International Conference on Inventive Computation Technologies (ICICT), 2026.

Kaushal Acharya 1 Sunil Ale 1 Rajan Kadel 2, Kaushal Acharya, Sunil Ale, Rajan Kadel · 2026

Read AI-Powered Image Analysis for Phishing Detection PubID: pubid: This is a preprint version of the paper accepted for the 9th International Conference on Inventive Computation Technologies (ICICT), 2026. by Kaushal Acharya 1 Sunil Ale 1 Rajan Kadel 2, Kaushal Acharya, Sunil Ale, Rajan Kadel (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-13555.txt.

Introduction

Acharya, Ale, and Kadel detect phishing sites from screenshots rather than text or web addresses, since fake pages copy logos, layouts, and colors. They compare two vision models, ConvNeXt-Tiny and a Vision Transformer, trained on webpage images with ImageNet starting weights. ConvNeXt-Tiny wins on balance between catches and false alarms and runs more cheaply, and the authors stress tuning the decision threshold for real deployment.

Catalog record

CREATORKaushal Acharya 1 Sunil Ale 1 Rajan Kadel 2, Kaushal Acharya, Sunil Ale, Rajan Kadel
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE33586
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.13555

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

AI-Powered Image Analysis for Phishing Detection PubID: pubid: This is a preprint version of the paper accepted for the 9th International Conference on Inventive Computation Technologies (ICICT), 2026. Kaushal Acharya 1 Sunil Ale 1 Rajan Kadel 2, Kaushal Acharya, Sunil Ale, Rajan Kadel Abstract Phishing websites now rely heavily on visual imitation—copied logos, similar layouts, and matching colours—to avoid detection by text- and URL-based systems. This paper presents a deep learning approach that uses webpage screenshots for image-based phishing detection. Two vision models, ConvNeXt-Tiny and Vision Transformer (ViT-Base), were tested to see how well they handle visually deceptive phishing pages. The framework covers dataset creation, preprocessing, transfer learning with ImageNet weights, and evaluation using different decision thresholds. The results show that ConvNeXt-Tiny performs the best overall, achieving the highest F1-score at the optimised threshold and running more efficiently than ViT-Base. This highlights the strength of convolutional models for visual phishing detection and shows why threshold tuning is important for real-world deployment. As future work, the curated dataset used in this study will be released to support reproducibility and encourage further research in this area. Unlike many existing studies that primarily report accuracy, this work places greater emphasis on threshold-aware evaluation to better reflect real-world deployment conditions. By examining precision, recall, and F1-score across different decision thresholds, the study identifies operating points that balance detection performance and false-alarm control. In addition, the side-by-side comparison of ConvNeXt-Tiny and ViT-Base under the same experimental setup offers practical insights into how convolutional and transformer-based architectures differ in robustness and computational efficiency for visual phishing detection. I Introduction One of the pervasive cybersecurity threats is phishing. It has exploited human trust through deceptive web interfaces and digital communications. When studying traditional phishing detection systems, they have relied mainly on textual content or URL-based analysis. New age attackers now employ and use visual deception logos, colour schemes, and layout structures to mimic and present themselves as legitimate websites. They also evade text- and domain-based filters. These evolving threats have made image-based phishing detection an area of research, as it is emerging as a critical frontier in cybersecurity [ 19 , 6 ] . The development and advancement of deep learning and computer vision in recent years have enabled automated extraction of spatial and contextual features. This is done by using webpage screenshots. Because of this, detection models are now capable of recognising and flagging subtle inconsistencies in visual composition. Convolutional Neural Networks (CNNs) and transformer-based vision architectures have shown and demonstrated the identification of such patterns. Most of the existing studies largely focus on accuracy but silently overlook and neglect other essential aspects, which are data set diversity, rigorous threshold optimisation, and model interpretability that are crucial for real-world deployment. This paper addresses these areas not addressed by the existing literature. To achieve it, we developed an AI-powered image analysis framework, which is capable of systematically evaluating and analysing modern vision architectures for advanced phishing detection. For this, we have selected two representative vision models, ConvNeXt-Tiny [ 10 ] and Vision Transformer (ViT-Base) [ 3 ] . We used these models to train and analyse their architectural strengths and feature extraction capabilities. Also, robustness against visually obfuscated phishing attacks. Selected models follow systematic processes in the order of dataset collection and construction, preprocessing, augmentation, model training, and threshold tuning and analysis for maximum output. Balancing recall and precision for deployment-relevant decision-making is the major result of threshold tuning, which is the objective of the paper. In summary, this study addresses the gap between the high reported accuracy in image-based phishing detection and the practical challenges of deploying such systems in real-world settings. While prior research has shown promising results, many studies rely primarily on accuracy and do not examine how models behave under different decision thresholds. The main purpose of this work is therefore to develop and evaluate a threshold-aware framework that provides a more realistic assessment of model performance. By systematically analysing precision, recall, and F1-score across varying thresholds and comparing ConvNeXt-Tiny and ViT-Base under the same experimental conditions, this study aims to offer clearer insight into both performance stability and practical applicability. The contributions of this paper are as follows: • A threshold-aware evaluation methodology that optimises precision, recall, and F1-score for real-world deployment. An experimental analysis highlighting architectural trade-offs between accuracy and computational efficiency • An end-to-end framework for image-based phishing detection using webpage screenshots, comparative evaluation of ConvNeXt-Tiny and transformer-based vision architectures; The remainder of this paper is structured as follows. Section II reviews related works on phishing detection. Section III defines the problem statement and outlines the contributions. Section IV describes the methodology used in this research. Section V presents the experimental evaluation and comparative analyses. Finally, Section VI concludes the paper with key findings and future directions. II Related Works This section reviews recent literature on phishing detection techniques across text, URL, and image modalities. II-A Text-based Phishing Detection Traditional phishing detection using text has primarily relied on linguistic and semantic features that were extracted from email bodies and subjects. Natural language processing, combined with feature engineering, enables classifiers to distinguish between legitimate and phishing messages with high accuracy. Some of the common techniques used were tokenisation, Term Frequency-Inverse Document Frequency (TF-IDF) weighting, lemmatisation, Chi-square, principal component analysis, and latent semantic analysis to generate compact but discriminative feature sets [ 4 ] . Classifiers such as XGBoost, Random Forest, and Support Vector Machines (SVMs) have demonstrated strong performance on benchmark datasets when trained on engineered text features [ 4 ] . Latest reviews have highlighted that TF-IDF, n-grams, and word embeddings are still dominant techniques in text-based phishing detection, along with new CNNs and Recurrent Neural Networks (RNNs) gaining adoption recently [ 15 ] . Verma et al. further showed that preprocessing techniques such as stemming, stopword removal, and n-gram extraction improve detection, though false positives persist due to overlaps with legitimate marketing emails [ 18 ] . Despite these advances, text-based models remain insufficient against modern attacks that embed logos, images, or obfuscated scripts, underscoring the need for multi-modal defences. TABLE I: Summary of Image-Based Phishing Detection Approaches and Datasets Method / Model Accuracy Dataset (Name & Size) Key Observation(s) CNN-based models [ 20 ] >95\% Balanced URL dataset (5,000 phishing + 5,000 legitimate) Dataset dependency issues CNN + LSTM hybrid [ 1 ] >97\% Public phishing benchmark datasets (size not explicitly reported) High recall; computationally expensive Phishpedia (Faster-RCNN + Siamese) [ 8 ] 99.2\% Phishpedia benchmark ( \sim 30K phishing + \sim 30K benign webpages) Detected 1700+ zero-day sites; runtime of 0.19 s/page CNN visual similarity [ 13 ] \sim 96\% Screenshot-based phishing datasets (e.g., Phish-IRIS \sim 2,852 images) Strong logo recognition; limited dataset diversity Vision GNN [ 9 ] >97\% Vision-based phishing benchmark datasets (dataset size not clearly specified) Captures relational structure among webpage elements Phish-IRIS (compact descriptors) [ 2 ] \sim 92\% Phish-IRIS dataset (1,313 training + 1,539 testing images) Lightweight approach; less accurate than CNNs II-B URL-based Phishing Detection URL-based phishing detection is one of the widely studied areas. It is studied using lexical, structural, and domain-related attributes. One of the recent studies applied a dataset of over 11,000 URLs with 33 extracted features, such

Read the complete text