Why Trust Your Agent? Empirical Security Gains from TRiSM-Guided Agentic Workflows in Healthcare
Liam Kearns · 2026
Read Why Trust Your Agent? Empirical Security Gains from TRiSM-Guided Agentic Workflows in Healthcare by Liam Kearns (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-28666.txt.
Introduction
Liam Kearns, writing alone, rebuilds a medical report-writing agent with security as the blueprint rather than the paint. Language-model agents often hold access rights beyond ordinary users, and bolting security on afterward invites data exposure and regulatory breaches. Kearns applies the AI TRiSM framework, covering trust, risk, and security management, to redesign the workflow.
Across five language models, two report types, hundreds of generations, and five hundred attack scenarios, the guided workflow cut attack success sharply. Poisoning of the retrieval store succeeded about a third as often, and data-field injection dropped from over two fifths to a quarter. The network injection route vanished entirely once prompts were built server-side. Accuracy rose at the same time, by fourteen points. Least privilege and defense in depth, the study concludes, buy reliability as well as safety.
Catalog record
| CREATOR | Liam Kearns |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 50 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2606.28666
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Why Trust Your Agent? Empirical Security Gains from TRiSM-Guided Agentic Workflows in Healthcare Liam Kearns Abstract Agent-based AI has enabled the automation of tasks by exposing application tools and resources to large language models (LLMs). However, to improve scope and accuracy, agents are often given access rights that exceed those of ordinary users, introducing significant security risks. AI is routinely integrated into applications with a disregard to security, risking data exposure and breaching regulations. This paper applies the AI Trust, Risk, and Security Management (TRiSM) framework to a medical report-generation application to demonstrate how an insecure agent workflow can be transformed into security-conscious agentic workflow. Both workflows were evaluated across five LLMs (Claude Haiku 4.5, GPT-4.1-nano, GPT-4.1-mini, GPT-5.4-mini, and Gemini 2.5 Flash) on two report types, totalling 800 generations and 500 attack scenarios including RAG poisoning, data-field injection, and client-side network injection. The TRiSM-guided agentic workflow reduced mean attack success rates from 31% to 10% for RAG poisoning and from 42% to 25% for data-field injection, while eliminating the network injection vector entirely through server-side prompt construction. Furthermore, report accuracy increased by 14 percentage points (72.5% to 86.5%) with the agentic workflow, demonstrating a secure design which provides more reliable outputs. This paper contributes to knowledge by demonstrating least-privilege, defence in depth agentic workflows improving security and accuracy, while also highlighting model choice is a necessary architectural consideration. K eywords agentic AI \cdot TRiSM \cdot prompt injection \cdot security 1 Introduction Agentic applications have become a popular way to infuse AI capabilities into projects. Agents, often powered by LLMs, work more effectively when provided with sufficient data sources and application tools. While this allows agentic AI to be used in high-value scopes, such as medical diagnosis, this can raise patient safety concerns due to the accountability and transparency of these automated decisions [ 2 ] . Not only does this risk exposure of sensitive data to the hosts of the AI models, but also risks exposure to malicious users who can intercept requests going to the AI and consequently the sensitive information being sent. Recent advances have expanded the capabilities of agentic workflows. Model Context Protocol (MCP) servers provide structured protocols in which agents can invoke tools and execute actions, while Retrieval-Augmented Generation (RAG) with vector knowledge bases gives agents access to context specific information at inference time. However, these components create a greater attack surface for agentic workflows. MCP tool logic can be redacted, making it difficult to map out security risks. Similarly, knowledge bases are not guaranteed to comply with the same access controls as application databases, increasing the risk of sensitive information being compromised. Integration of AI into projects must go through appropriate frameworks to ensure risks are managed. Without managing risks, insecure development practices, such as inexperienced vibe coding, whereby AI-generated code is integrated with minimal review, can expose agents to be exploited through risks from injection and insecure design with vibe coding producing functional but insecure code [ 32 ] . To control such risks, frameworks such as AI Trust, Risk, and Security Management (TRiSM) can map out model governance and trustworthiness. This paper investigates the vulnerabilities introduced by improper agent integration and how these risks can be reduced through TRiSM-guided workflows. Security challenges from the complexity of agent execution logic have been highlighted by previous research [ 4 ] , but steps to mitigate these vulnerabilities can be under-represented. Additionally, the risks of using single-agent workflows for agentic AI task introduces risks of overwhelming the underlying LLM, increasing chances of malicious prompts succeeding. The motivation to design the TRiSM-guided agentic workflow came from existing application architecture using basic REST API logic to interact with LLMs with minimal guardrails, risking the credibility of AI outputs and the security of the application as a whole. The scope of this paper is focused on using AI agents to generate medical reports in a medical application. Figure 1 illustrates clinical encounter information from the medical application using agents, tools, and resources to generate medical reports for standard and Covid-19 encounters. This application uses LLM generative capabilities to also generate recommended treatments if not provided by the attending physician. This scope provides a clear scenario to implement a secure agentic workflow to protect the application and patient data alongside the accuracy of generated reports. Figure 1: Medical application using AI agents to generate medical reports. The structure of the paper is as follows. Section 2 investigates significant attack vectors that AI-powered applications need to defend against. Section 3 outlines the insecure agent workflow and TRiSM-guided agentic workflow as well as efficiency metrics. Next, Section 4 discusses the vulnerabilities and performance of both workflows across various LLM models. Subsequently, Section 5 outlines the final recommended agentic workflow which is influenced by TRiSM principles. 1.1 Contributions In this paper, vulnerabilities in AI-powered apps are identified and controlled through risk management. The paper’s major contributions are as followed: • Applying TRiSM principles to an operational medical agentic application, showing which concrete architectural controls each principle maps to. • Demonstrating that the TRiSM-guided redesign, of which the multi-agent workflow is a consequence, reduces attack success across three vectors and five models while improving report accuracy. • Identifying the vulnerability classes that emerge when LLM agents are granted access beyond the underlying user’s permissions, with per-model attack-success benchmarks. 2 Related works AI capabilities can be implemented in various ways within applications, each introducing its own risks. This section reviews the vulnerabilities most relevant to LLM-integrated medical applications and shows that existing mitigations address each vulnerability in isolation, but rarely in a combined architectural response for a medical application. Additionally, the AI Trust, Risk, and Security Management (TRiSM) framework is introduced that aims to mitigate these risks. 2.1 Injection attacks Prompts sent to LLMs can be manipulated without attackers having access to model gradients or weights. These prompt injection attacks can be used for prompt leaking or code generation with a high success rate in LLM-integrated applications [ 17 ] . Even when mitigated against through instructing the output to be a specific phrase or format (known-answer detection), LLMs used in decision making can still be heavily influenced by prompt injection [ 24 ] . This highlights how injection attacks can impact LLM evaluations and bypass detection techniques in the output. Injection attacks can be mitigated against by enforcing input and output formats and time constraints on outputs [ 17 ] . However, indirect injection attacks can occur through attackers injecting prompts into data which will be used to build prompts [ 7 ] . Injection opportunities increase significantly when using RAG, where an attacker can subtly change semantics to introduce bias into retrieved information [ 6 ] . With data and instructions both being used in prompts, injection attacks can be the consequence of traditional data integrity attacks such as data tampering and SQL injections. For medical applications, injection attacks can pose danger to patient confidentiality and physical wellbeing. RAG is often used in medical scenarios to provide a ground truth, but clinically phrased injections can influence treatment decisions or fabricate evidence [ 15 ] . This highlights the need for safety controls at the orchestration layer of AI-powered workflows rather than expecting models