Detecting Undesired Process Behavior by Means of Retrieval Augmented Generation
Michael Grohs, Adrian Rebmann, Jana-Rebecca Rehse · 2025
Read Detecting Undesired Process Behavior by Means of Retrieval Augmented Generation by Michael Grohs, Adrian Rebmann, Jana-Rebecca Rehse (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-22041.txt.
Introduction
Michael Grohs, Adrian Rebmann, and Jana-Rebecca Rehse propose a way to spot unwanted behavior in business processes, in a 2025 paper. Standard checks compare recorded process runs with a formal model of good behavior. When no model exists, those checks cannot run.
Other teams fine-tune large language models for the task, which costs much and transfers poorly. Grohs and co-workers instead give the model direct access to a store of good and bad examples from other processes, using retrieval. In their tests this beat fine-tuned models, above all when event log context was added. The work offers a cheaper path for firms without process models.
Catalog record
| CREATOR | Michael Grohs, Adrian Rebmann, Jana-Rebecca Rehse |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 52 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-22041
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Detecting Undesired Process Behavior by Means of Retrieval Augmented Generation Michael Grohs, Adrian Rebmann, Jana-Rebecca Rehse Abstract Conformance checking techniques detect undesired process behavior by comparing process executions that are recorded in event logs to desired behavior that is captured in a dedicated process model. If such models are not available, conformance checking techniques are not applicable, but organizations might still be interested in detecting undesired behavior in their processes. To enable this, existing approaches use Large Language Models (LLMs), assuming that they can learn to distinguish desired from undesired behavior through fine-tuning. However, fine-tuning is highly resource-intensive and the fine-tuned LLMs often do not generalize well. To address these limitations, we propose an approach that requires neither a dedicated process model nor resource-intensive fine-tuning to detect undesired process behavior. Instead, we use Retrieval Augmented Generation (RAG) to provide an LLM with direct access to a knowledge base that contains both desired and undesired process behavior from other processes, assuming that the LLM can transfer this knowledge to the process at hand. Our evaluation shows that our approach outperforms fine-tuned LLMs in detecting undesired behavior, demonstrating that RAG is a viable alternative to resource-intensive fine-tuning, particularly when enriched with relevant context from the event log, such as frequent traces and activities. 1 Introduction Conformance checking is a sub-discipline of process mining which compares process executions that are captured as traces recorded in an event log to process behavior captured in a process model [ 9 ] . As input, conformance checking techniques require an event log and a dedicated process model that explicitly specifies what behavior is desired in the process [ 7 ] . Based on this, they can identify which traces diverge from the desired behavior and, subsequently, locate the undesired behavior within the trace [ 7 ] . This undesired behavior has negative impact on any objective of the process, including but not limited to compliance, performance, and outcome. In this paper, we focus on undesired control-flow of processes, i.e., activity executions that are not envisioned. The applicability of conformance checking hence depends on the availability of high-quality models for the process in question [ 21 , p.11] . However, creating and maintaining such models requires substantial effort and stakeholder involvement [ 11 ] , which means that they are often unavailable, incorrect, or out of date [ 9 ] . Further, the process at hand might be too complex to be effectively captured in a model [ 2 ] or the organization might decide not to model its processes to spare itself the efforts [ 11 ] . In all of these cases, conformance checking is not applicable. Nevertheless, organizations might still want to know whether their processes contain undesired behavior [ 8 ] . To alleviate this issue, several approaches have been proposed that detect undesired behavior in event logs without a dedicated process model. Unsupervised anomaly detection techniques identify statistically infrequent traces, assuming these are undesired [ 22 , 17 ] . However, since low frequency does not always indicate undesirability, some approaches instead leverage the natural language semantics of activities to detect undesired process behavior [ 1 ] . Recent approaches leverage the natural language understanding capabilities of Large Language Models (LLMs) for this task. The key idea is that fine-tuning an LLM on process behavior enables it to learn how processes function, allowing it to differentiate between desired and undesired behavior [ 6 , 8 , 15 ] . However, fine-tuning is highly resource-intensive and the resulting models often do not generalize well, especially to processes that differ considerably from those used for fine-tuning [ 6 ] . To overcome these limitations, we propose an approach for detecting undesired process behavior that requires neither a dedicated process model nor resource-intensive fine-tuning. Instead, we use Retrieval Augmented Generation (RAG), which builds on the idea that LLMs answer more accurately if they are supplied with explicit task-related knowledge that has not been used in training [ 19 ] . Concretely, we provide an LLM with direct access to a knowledge base that captures both desired and undesired behavior of a broad range of processes. In addition, we provide it with explicit knowledge about the event log at hand in form of frequent traces and activities. Then, the LLM can use this task-related knowledge to find undesired behavior in previously unseen traces. In particular, our approach can detect instances of five established patterns of undesired behavior [ 16 ] : inserted , skipped , repeated , replaced , and swapped activities (or sequences thereof). Thus, our approach can identify more complex instances of undesired behavior compared to existing approaches, which are limited to deviations involving at most two activities [ 6 , 8 ] , even though undesired behavior can span any number of activities. Our evaluation experiments show that our approach outperforms both general-purpose LLMs and LLMs that have been fine-tuned for detecting undesired process behavior. The remainder of this paper is structured as follows. We first discuss related work in Section 2 . Then, we present our approach in Section 3 and evaluate it in Section 4 . We discuss our findings and conclude in Section 5 . 2 Related Works Our work relates to conformance checking, unsupervised anomaly detection, and semantic anomaly detection. Also, it relates to RAG-usage in other process mining tasks. Conformance Checking. Traditional conformance checking techniques compare event log traces to process models that define desired process behavior [ 7 ] . They all require a process model as input [ 9 ] . State-of-the-art techniques are trace alignments [ 9 ] , which identify inserted and missing activities. Based in these alignments, it is also possible to identify high-level patterns of undesired behavior such as repetitions or swaps [ 12 , 13 ] . Furthermore, declarative conformance checking techniques check the satisfaction of constraints, identifying violations of temporal patterns like precedence or response [ 18 ] . Unlike our approach, all corresponding techniques require a process model as input. Unsupervised Anomaly Detection. Unsupervised anomaly detection tries to find undesired behavior without relying on a process model, by identifying statistically rare behavior in a given event log. Corresponding techniques are, e.g., based on association rule mining [ 5 ] or likelihood graphs of traces [ 4 ] . Deep learning methods like GRU autoencoder neural networks combined with an anomaly threshold [ 22 ] or LSTM networks [ 17 ] have also been proposed. All these techniques determine whether a process instance is statistically unlikely. However, likeliness does not necessarily correspond to desirability, so infrequency alone does not suffice to detect deviations from desired process behavior. This constitutes a conceptual difference to the goal of our approach. Semantic Anomaly Detection. Rather than relying on infrequency, several approaches employ language models, which allows them to consider the natural language semantics of activities to identify undesirable behavior. This has been done on trace-level and activity-level. On trace-level, semantically anomalous traces can be detected [ 24 ] , but this does not allow for locating undesired activities. On activity-level, some approaches identify semantically anomalous ordering relationships between pairs of activities [ 8 , 24 ] , but can therefore not detect more