FraudTransformer: Time-Aware GPT for Transaction Fraud Detection AI for Financial Fraud Detection and Prevention CCS: Computing methodologies Neural networks CCS: Computing methodologies Machine learning CCS: Information systems Data mining CCS: Security and privacy Intrusion/anomaly detection and m

Gholamali Aminian, Andrew Elliott, Tiger Li, Timothy Cheuk Hin Wong, Victor Claude Dehon, Łukasz Szpruch, Carsten Maple, Christopher Read, Martin Brown, Gesine Reinert, Mo Mamouei · 2025

Read FraudTransformer: Time-Aware GPT for Transaction Fraud Detection AI for Financial Fraud Detection and Prevention CCS: Computing methodologies Neural networks CCS: Computing methodologies Machine learning CCS: Information systems Data mining CCS: Security and privacy Intrusion/anomaly detection and m by Gholamali Aminian, Andrew Elliott, Tiger Li, Timothy Cheuk Hin Wong, Victor Claude Dehon, Łukasz Szpruch, Carsten Maple, Christopher Read, Martin Brown, Gesine Reinert, Mo Mamouei (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-23712.txt.

Introduction

Payment fraud hides inside streams where both the order of events and the gaps between them carry clues. Gholamali Aminian and 10 co-authors built FraudTransformer, a GPT-style sequence model shaped for transaction streams. A dedicated time encoder embeds absolute timestamps or gaps between events. A learned positional encoder keeps the relative order of the stream intact.

On an industrial data set with tens of millions of transactions and side events, the model beat four strong classical baselines, including gradient boosting tools. It also beat transformer variants missing either the time or the position part. On held-out data it posted the best AUROC and PRAUC scores.

Catalog record

CREATORGholamali Aminian, Andrew Elliott, Tiger Li, Timothy Cheuk Hin Wong, Victor Claude Dehon, Łukasz Szpruch, Carsten Maple, Christopher Read, Martin Brown, Gesine Reinert, Mo Mamouei
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE34 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2509-23712

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

FraudTransformer: Time-Aware GPT for Transaction Fraud Detection AI for Financial Fraud Detection and Prevention CCS: Computing methodologies Neural networks CCS: Computing methodologies Machine learning CCS: Information systems Data mining CCS: Security and privacy Intrusion/anomaly detection and malware mitigation Gholamali Aminian, Andrew Elliott, Tiger Li, Timothy Cheuk Hin Wong, Victor Claude Dehon, Łukasz Szpruch, Carsten Maple, Christopher Read, Martin Brown, Gesine Reinert, Mo Mamouei Abstract . Detecting payment fraud in real-world banking streams requires models that can exploit both the order of events and the irregular time gaps between them. We introduce FraudTransformer, a sequence model that augments a vanilla GPT-style architecture with (i) a dedicated time encoder that embeds either absolute timestamps or inter-event values, and (ii) a learned positional encoder that preserves relative order. Experiments on a large industrial dataset—tens of millions of transactions and auxiliary events—show that FraudTransformer surpasses four strong classical baselines (Logistic Regression, XGBoost and LightGBM) as well as transformer ablations that omit either the time or positional component. On the held-out test set it delivers the highest AUROC and PRAUC. 1. Introduction Fraudulent activities, such as financial fraud, e-commerce scams, and identity theft, are becoming increasingly sophisticated, necessitating the development of advanced detection methods ( hilal2022financial ) . Conventional approaches rely on handcrafted features and while can be effective in some cases, often fail to identify nuanced patterns of fraudulent behaviour, especially in dynamic and high-dimensional datasets. There is promise in using an automated feature creation methods such as those provided by a transformer architecture. Such an architecture should be able to identify novel subtle patterns before even human monitors detect them. However transformer architectures cannot easily detect temporal information ( zeng2023transformers ) , potentially limiting their utility for fraud detection, as this is a time-sensitive application. In this paper we assess the combination of a GPT architecture with time-encoders to provide time embeddings on a dataset of financial transactions. The time-encoders used here are rotational and sinusoidal embeddings combined with a positional encoder; the embeddings can be based on absolute time, or on the relative time between events. We evaluate their ability to improve detection of frauds on a dataset of financial transaction which is held by a national HSBC Payment Fraud Group. These large-scale real-world data consist of temporal patterns of customers’ digital journeys and payments. While the limited scale and granularity of public synthetic datasets restricts the generalisability of findings based on them to real-world settings, the data used in this study naturally incorporates the noise, outliers and irregularities that are common in production systems. Our key findings are as follows. • Using time-encoder with a lightweight LayerNorm-based integration of the time embeddings can improve the performance of a model for fraud-detection tasks. • Event-level relative sinusoidal embeddings paired with a learned positional encoder deliver the highest performance on the test set, outperforming three classical baselines. This comparison varies the timestamp scheme (absolute vs relative), the encoder (sinusoidal vs rotary), and the presence or absence of a positional encoder. Based on these findings we propose an algorithm which we call FraudTransformer, combining a vanilla GPT architecture with relative sinusoidal embeddings. This paper is organized as follows. Section 2 surveys related work on transformer-based fraud detection. Section 3 defines the problem. Section 6 details our FraudTransformer algorithm and GPT model. Section 4 describes the dataset, and Section 7 presents the experiments and results. Section 8 concludes and outlines directions for future work. Hyperparameter tuning results are deferred to the Appendix. 2. Related Works Fraud detection methods typically fall into two main groups: classical supervised learning on tabular transaction data and deep learning for tabular inputs For tabular supervised learning, gradient-boosted decision trees—most notably XGBoost and LightGBM—are strong, low-latency baselines because they exploit hand-engineered behavioral features; LightGBM remains a standard reference point for card and e-commerce fraud tasks ( ke2017lightgbm ) . In tabular deep learning, multilayer perceptrons (MLPs) and autoencoders are common supervised and reconstruction-based options, while specialized models such as TabNet, which uses sequential feature attention to improve interpretability and often narrows the gap to boosted trees ( arik2021tabnet ) , and TabTransformer, which applies self-attention to categorical embeddings ( huang2020tabtransformer ) , have gained traction. In this work, we focus on transformers applications for fraud detection. Transformer architectures have revolutionised the field of machine learning over the last 10 years. While transformer models are computationally expensive to train, once trained they can be deployed in real time. Fundamental to the advances of this approach is so-called attention, the ability of this architecture when considering a large sequence of inputs to decide what to pay attention to. Its strengths make transformers suitable for fraud detection (e.g. see ( zhang2023fata ; padhi2021tabular ; hu2023bert4eth ; yang2023finchain ) ). Indeed, transformer architectures are increasingly central to fraud detection research, with investigations spanning encoder-only, decoder-only, and hierarchical designs. Encoder-type models, like BERT, are typically adapted through transfer learning; for instance, ( taneja2025fraud ) showed that fine-tuning a pre-trained BERT on an online-recruitment corpus led to substantial gains in detection accuracy. Similarly, decoder-only transformers have been explored by tokenizing transactional events and pre-training GPT models on these sequences, leveraging their autoregressive nature to flag anomalous patterns, as demonstrated in ( stein2024simple ) . To capture richer structures in tabular data, hierarchical variants have been proposed, such as TabFormer, a BERT-based architecture that embeds individual transactions before aggregating them into table-level representations, achieving state-of-the-art performance ( padhi2021tabular ) . Building on this, ( zhang2023fata ) proposed FATA-trans to incorporate static fields—constant across all transactions—via separate encoders, further enhancing the ability of the model to detect fraud. In the FATA-trans structure, inspired by sinusoidal positional embedding, a time-aware positional embedding based on absolute timestamps of transactions is added to embedding of transactions. Motivated by the success of generative pretrained transformers (GPT) for large language models, GPT architectures have also been employed for social security fraud detection ( jin2025large ) . A more detailed survey on transformer-based anomaly detection can be found for example in ( ma2024research ) . To the best of our knowledge, using the relative time at event level in combination with a GPT model for financial fraud detection tasks has not yet been proposed. This paper fills that gap. 3. Problem Statement Fraud detection requires capturing complex temporal and contextual dependencies across various data modalities, such as transactional metadata, user behaviour, and timestamps. Traditional machine learning models often struggle to process such intricate relationships. While a GPT architecture is fast, by default all tokens are assigned the same importance. Although attention mechanisms are learnt to focus on the most important features, empirically, learning time and position-dependence in features is a challenge for GPT models ( radford2018improving ) . To address this issue, so-called positional encoders have been proposed. Positional encodings aid transformers in utilising positional information, originally regarding words in sequences. There are two widely used positional encoding schemes for transformers, • Sinusoidal positional encoding ( vaswani2017attention ) ; • Rotary positional encoding ( su2024roformer ) . A GPT transformer does not have an intrinsic notion of time and hence detecting time-dependent patterns tends to be difficult. To give the model temporal awareness, we use positional encoders but now as time encoders, replacing the usual

Read the complete text