ReGA: Model-Based Safeguard for LLMs via Representation-Guided Abstraction
Zeming Wei, Chengcan Wu, Meng Sun · 2025
Read ReGA: Model-Based Safeguard for LLMs via Representation-Guided Abstraction by Zeming Wei, Chengcan Wu, Meng Sun (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-01770.txt.
Introduction
Zeming Wei, Chengcan Wu, and Meng Sun present ReGA, a guard system that watches a language model's inner states for signs of harmful requests, in a 2025 paper. Classic model-checking tools cannot scale to models this large. Recent work, though, shows safety ideas live along a few key directions in the model's hidden layers.
ReGA builds a small abstract model around those safety directions, then uses it to judge prompts and conversations. It scored near 0.98 on a standard ranking measure at the prompt level and slightly higher across whole conversations. The authors frame representation-guided abstraction as a way to keep formal safety checks alive in the era of giant models.
Catalog record
| CREATOR | Zeming Wei, Chengcan Wu, Meng Sun |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 67 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2506-01770
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
ReGA: Model-Based Safeguard for LLMs via Representation-Guided Abstraction Zeming Wei, Chengcan Wu, Meng Sun Abstract . Large Language Models (LLMs) have achieved tremendous success in various tasks, yet concerns about their safety and security have emerged. In particular, they pose risks of generating harmful content and are vulnerable to jailbreaking attacks, creating unaddressed security issues regarding their deployments. In the context of software engineering for artificial intelligence (SE4AI) techniques, model-based analysis has demonstrated notable potential for analyzing and monitoring machine learning models, particularly in stateful deep neural networks. However, it suffers from scalability issues when extended to LLMs due to their vast feature spaces. In this paper, we aim to address the scalability issue of model-based analysis techniques for safeguarding LLM-scale models. Motivated by the recent discovery of low-dimensional safety-critical representations that emerged in LLMs, we propose ReGA, a model-based analysis framework with Representation-Guided Abstraction, to safeguard LLMs against harmful prompts and generations. By leveraging safety-critical representations, which are key directions in hidden states that indicate safety-related concepts, ReGA effectively narrows the scalability gap when developing the abstract model for safety modeling. Our comprehensive evaluation shows that ReGA performs sufficiently well in distinguishing between safe and harmful inputs, achieving an AUROC of 0.975 at the prompt level and 0.985 at the conversation level. Additionally, ReGA exhibits robustness to real-world attacks and generalization across different safety perspectives, outperforming existing safeguard paradigms in terms of interpretability and scalability. Overall, ReGA serves as an efficient and scalable solution to enhance LLM safety by integrating representation engineering with model-based abstraction, paving the way for new paradigms to utilize software insights for AI safety. Our code is available at https://github.com/weizeming/ReGA . DOI: 10.1145/3808177 Journal: PACMSE Volume: 3 FSE FSE170 7 fse26mainb-p1401-p CCS: Security and privacy Software security engineering CCS: Computing methodologies Natural language generation † † cc-license: by 1. Introduction In the past few years, Large Language Models (LLMs) have achieved remarkable success across various tasks, significantly transforming the paradigms of machine learning (ML). By leveraging knowledge from extensive data sets and utilizing advanced next-token decoding strategies, LLMs have made significant advancements in areas such as chat completion [ 42 , 33 ] , mathematical reasoning [ 27 , 2 ] , code generation [ 24 , 16 ] , and program repair [ 8 , 32 ] tasks. These accomplishments established the foundational role of LLMs in modern ML and software systems. However, like traditional ML models and software, current LLMs face significant trustworthiness issues related to their safety and security [ 4 , 85 , 80 , 12 ] . While their advanced reasoning and generation capabilities have facilitated numerous applications, concerns about their potential to produce harmful or toxic content have also emerged. Various evaluations on LLMs have shown that LLMs may positively respond to unsafe requests from users that violate their safety or ethical guidelines [ 38 , 61 , 26 , 86 ] . Furthermore, recent attack techniques have been developed to deceive LLMs into generating harmful content, even if the LLMs successfully reject the request prompt in its original form. These tactics are known as jailbreaking attacks [ 52 , 93 , 66 , 35 ] , raising further security concerns about LLMs. To prevent confusion, this work refers to the security risk as the generation of harmful content by LLMs. To mitigate such concerns, numerous efforts have been dedicated to aligning LLMs with human values during their pre-training or post-training phases [ 33 , 6 , 18 ] . Nevertheless, safeguarding LLMs against unsafe prompts and outputs in an effective and efficient manner remains a challenging problem [ 4 ] . Notably, model-based analysis techniques have demonstrated strong potential in analyzing, inspecting, and repairing Deep Neural Networks (DNNs) through a thread of recent research [ 21 , 76 , 22 , 48 , 77 , 45 , 70 ] . For example, DeepStellar [ 21 ] constructs an abstract model to detect adversarial inputs of recurrent neural networks (RNNs), and Marble [ 22 ] analyzes the robustness of stateful DNNs through similar model abstraction. By constructing more formal and interpretable abstract models that simulate specific properties of the target DNN with internal hidden-state feature extraction, these works successfully achieve various analysis goals related to the DNN. So far, model-based analysis has become a representative paradigm of leveraging software techniques into DNN analysis [ 25 , 37 ] , contributing significantly to the success of software engineering for artificial intelligence (SE4AI) research. However, existing abstract model extraction-based techniques suffer from scalability hurdles when extending to LLM-scale models, limiting their effectiveness to LLMs with architectures and parameters much larger than those of traditional DNN models. In particular, a recent preliminary work, LUNA [ 55 ] , explores universal analysis on LLMs through extracting abstract models like Discrete Time Markov Chain (DTMC), but requires a vast amount of abstract states to be effective (ranging from hundreds to millions), limiting the efficiency and effectiveness of the abstract model for analysis. The fundamental bottleneck along this thread is that the feature space of LLMs is too large to be universally modeled by abstract models for different perspectives on trustworthiness. In this context, we refer to the scalability issue of model-based analysis as the effectiveness of their modeling when assessing the target property ( e.g. , security) in LLM-scale models. Figure 1 . The outline of ReGA. In this paper, we aim to address the scalability gap in existing model-based analysis techniques for LLM security safeguarding. Our main motivation is that, when concentrating on a specific perspective like safety, we can identify and extract information from the hidden states that are solely related to our focus, thus achieving effective dimension reduction. This particular focus is fundamentally different from existing model-based analysis frameworks like LUNA [ 55 ] that consider hidden states from all neurons and features, which results in significant computational complexity. To achieve this, we utilize the unique representations [ 92 ] that emerge in LLMs for constructing the abstract model to overcome the scalability issues for safeguarding LLMs. These representations consist of specific directions in hidden states that indicate particular concepts and are structured using concept-specific contrastive datasets. In the context of LLMs, the term representation is fundamentally different from features , where the latter usually indicates the overall hidden states. In particular, the safety-critical representations [ 67 , 88 , 84 ] denote the safety-related concepts within the inputs. Based on this motivation, we explore model-based analysis for LLMs against unsafe prompts or generated content, and propose ReGA, a Re presentation- G uided A bstraction framework for safeguarding white-box LLMs. As outlined in Figure 1 , the ReGA pipeline consists of three key phases. First, we sample safety representations from the LLM using a contrastive dataset that models safety-critical concepts (Phase I). Next, we apply unsupervised machine learning techniques to cluster these representations into abstract states and model their transition probabilities. Meanwhile, we bind the safety semantics from the contrastive dataset to construct an abstract model (Phase II). Finally, during inference, this abstract model safeguards the LLM at both the prompt level (before generation) and the conversation level (after generation) in Phase III. To validate the effectiveness of ReGA, we conduct