Rethinking Fraud Safety Evaluation: Multi-Round Attacks Reveal Safety–Utility Tradeoffs in Graph-Context LLM Defenders

Laura Jiang Curtin University [email protected] Reza Ryan Curtin University [email protected] Qian Li Curtin University [email protected] Nasim Ferdosian Curtin University [email protected] · 2026

Read Rethinking Fraud Safety Evaluation: Multi-Round Attacks Reveal Safety–Utility Tradeoffs in Graph-Context LLM Defenders by Laura Jiang Curtin University [email protected] Reza Ryan Curtin University [email protected] Qian Li Curtin University [email protected] Nasim Ferdosian Curtin University [email protected] (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-20759.txt.

Introduction

Laura Jiang and three colleagues test fraud defenders under multi-round attacks, where attackers escalate over several turns. Single-turn safety tests poorly reflect real fraud defence, and the paper asks when a defender declines, not just whether it ever does. Graph-context defenders refuse safely earlier than text-only baselines under replay and adaptive pressure. They also over-refuse benign cases far more, a cost traced to how the defender language model consumes structured graph fields.

Catalog record

CREATORLaura Jiang Curtin University [email protected] Reza Ryan Curtin University [email protected] Qian Li Curtin University [email protected] Nasim Ferdosian Curtin University [email protected]
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE66613
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.20759

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Rethinking Fraud Safety Evaluation: Multi-Round Attacks Reveal Safety–Utility Tradeoffs in Graph-Context LLM Defenders Laura Jiang Curtin University [email protected] Reza Ryan Curtin University [email protected] Qian Li Curtin University [email protected] Nasim Ferdosian Curtin University [email protected] Abstract Single-turn safety evaluation is a poor proxy for real fraud defense, where attackers escalate across multiple rounds. This paper evaluates fraud defenders under replay and adaptive multi-round attacks and measures when a defender refuses, not just whether it eventually refuses. On a frozen multi-round suite built from Fraud-R1, graph-context defenders improve early safe refusal relative to text-only baselines under both replay and adaptive fraud pressure, but they also produce substantially more benign over-refusal. Direct probing of the trained graph encoder, together with paired shuffle-risk ablations on both fraud and benign sides replicated across two seeds on the Qwen-1.5B backbone, localises this cost to how the defender LLM consumes structured context rather than to graph-encoder quality: the encoder cleanly separates fraud from benign, while the LLM responds primarily to the presence of structured graph fields and only secondarily, and asymmetrically, to risk-score magnitude. Temporal graph context is directionally stronger than static and significantly better grounded, but is not yet conclusively superior on the main refusal metrics. The contribution is evaluative and measurement-oriented: robust fraud assessment must be multi-round, must report refusal timing, must account for benign false positives alongside fraud-side safety gains, and must localize observed costs to the graph signal or to how the LLM consumes it. 1 Introduction Fraud is not a one-shot safety problem. Attackers build legitimacy, delay explicit requests, respond to hesitation, and reuse relational infrastructure across accounts, organizations, and channels. A defender that looks strong on a single malicious prompt may behave very differently once the attacker is allowed to continue the conversation or adapt to the defender’s previous action. This gap matters because a deployed fraud-defense assistant is judged not only by whether it eventually refuses, but by whether it refuses early enough and without rejecting benign traffic at an unusable rate. Large language models are increasingly deployed as decision-making assistants and conversational agents in financial and advisory contexts [ 1 , 2 ] , raising the stakes for fraud-defense robustness beyond what standard single-turn benchmarks can reveal. This paper is built around a simple claim: single-turn evaluation under-measures fraud safety behavior . Fraud defense is therefore evaluated under replay and adaptive multi-round conversations, measuring early safe refusal rather than eventual refusal alone. Graph-context prompting serves as a case study because fraud is relational as well as linguistic. The central question is not whether a graph model can beat a text model on a leaderboard. The question is what a more realistic evaluation regime reveals about timing, robustness, and false-positive cost. The evidence in this paper supports three observations, each anchored to specific frozen-suite measurements reported in Sections 6 and 8 . First, multi-round evaluation changes the picture relative to single-turn evaluation: on the frozen suite, single-turn text-only AUSR is 0.775 while replay text-only AUSR is 0.847 , and the multi-round protocol additionally exposes latency and grounding differences not visible at the single-turn level (Table 2 ). Second, graph context improves fraud-side early refusal over text-only prompting: under replay attack, temporal graph context raises AUSR from 0.847 to 0.978 ( p=0.0004 , paired permutation test of Algorithm 3 on n=80 pooled paired cases). Third, those gains come with substantial benign over-refusal: on benign replay traffic, ORR rises from 0.36 under text-only to 0.84 – 0.89 under graph context (Table 3 ). Together, these results support a measurement and insight paper more strongly than a pure method paper. This paper makes five contributions: 1. Fraud defense is formulated as a multi-round safety evaluation problem with replay and adaptive attackers rather than a fixed single-turn classification problem. 2. Refusal timing is established as a first-class robustness quantity; evaluation uses early safe refusal curves and area-under-curve metrics. 3. Empirical results show that graph-context defenders improve fraud-side early refusal relative to text-only baselines, but also incur large benign over-refusal costs that single-turn fraud evaluation does not expose. 4. A forward-looking design recommendation is offered for graph-context defenders: temporal graph context is worth pursuing in future work because of its consistent and statistically significant grounding advantage over static graph context, even though directional refusal gains do not reach significance on the current frozen suite. 5. The safety–utility tradeoff is localized to LLM context-consumption rather than graph-encoder quality. Direct probing of the trained encoder, together with a shuffled-risk ablation, indicates that the defender LLM responds primarily to the presence of structured graph context rather than to per-case risk-score magnitude. Paper organisation. The remainder of the paper is organised as follows. Section 2 surveys related work in fraud detection, LLM safety, graph–LLM integration, and concurrent multi-turn safety benchmarks. Section 3 formalises the multi-round fraud-defence problem and the proposed graph-context pipeline, including the exact prompt template that integrates serialized graph fields with the conversation history. Section 4 describes the attack regimes, defender contexts, metrics (including the grounding-score rubric), and the paired permutation test and bootstrap confidence-interval procedure used throughout. Section 5 fixes the frozen experimental suite from which all main numbers are drawn. Section 6 reports the multi-round, graph-context-helps, and benign-cost results. Section 8 reports targeted follow-up analyses: localisation of the safety–utility tradeoff to LLM context consumption, scale-consistency, and prompt-calibration. Section 9 translates these findings into concrete design recommendations for graph-context defenders, and Sections 10 and 11 discuss limitations and conclude. 2 Related Work 2.1 Fraud Evaluation and Fraud Detection Financial fraud detection has long relied on transaction-level classification and anomaly detection, but graph-based work has shown that coordinated behavior often becomes visible only through relationships among accounts, devices, domains, and counterparties [ 3 , 4 , 5 , 6 ] . Temporal variants further suggest that recency, repeated contact, and stage transitions carry useful fraud signal [ 7 , 8 , 9 , 10 , 11 ] . Benchmarks such as DetoxBench have extended fraud evaluation to multi-task abuse detection settings [ 12 ] . Fraud-R1 is especially important because it shifts the benchmark target from isolated malicious prompts to multi-round fraud inducement [ 13 ] . This work builds directly on that shift, but focuses on what such an evaluation setting reveals about safety measurement. 2.2 LLM Safety and Conversation Evaluation Large-language-model safety work has emphasized the importance of adaptive adversaries and conversation-level evaluation rather than single-turn prompts [ 14 , 15 , 16 , 17 , 18 ] . Alignment techniques such as RLHF and Constitutional AI have improved model behavior on standard safety benchmarks [ 2 , 19 ] , yet systematic trustworthiness evaluations reveal persistent gaps even in well-aligned models [ 20 , 21 ] . Prompt injection and indirect manipulation of LLM-integrated applications represent an additional attack surface [ 22 , 23 ] that is especially relevant when fraud attackers can control the input channel. However, most of that literature is not specialized to fraud, where persuasion is delayed, trust-building is explicit, and benign service communication can look superficially similar to malicious coordination. This makes fraud a good setting for studying not only whether a model refuses, but how refusal timing changes under realistic attack pressure. 2.3 Graph Context and Graph–LLM Integration Graph

Read the complete text