Astra : Autonomous Spatial-Temporal Red-teaming for AI Software Assistants

Xiangzhe Xu, Guangyu Shen, Zian Su, Siyuan Cheng, Hanxi Guo, Lu Yan, Xuan Chen, Jiasheng Jiang, Xiaolong Jin, Chengpeng Wang, Zhuo Zhang, Xiangyu Zhang · 2025

Read Astra : Autonomous Spatial-Temporal Red-teaming for AI Software Assistants by Xiangzhe Xu, Guangyu Shen, Zian Su, Siyuan Cheng, Hanxi Guo, Lu Yan, Xuan Chen, Jiasheng Jiang, Xiaolong Jin, Chengpeng Wang, Zhuo Zhang, Xiangyu Zhang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-03936.txt.

Introduction

Xiangzhe Xu and a large team present ASTRA, an agent that red-teams AI coding assistants on its own. Tools like Copilot write code and give security advice, yet their safety is poorly mapped. Fixed test sets miss real-world flaws. Their 2025 system works in three stages. First it builds knowledge graphs of software tasks and known weaknesses. Then it probes target models along two axes: what they are asked and how they reason.

The probes use requests a real developer might type. Hits become polished violation cases. Across two test domains ASTRA found far more issues than rival methods. Training on its cases also made models safer by a clear margin. The team pitches it as working gear for safer AI helpers.

Catalog record

CREATORXiangzhe Xu, Guangyu Shen, Zian Su, Siyuan Cheng, Hanxi Guo, Lu Yan, Xuan Chen, Jiasheng Jiang, Xiaolong Jin, Chengpeng Wang, Zhuo Zhang, Xiangyu Zhang
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE86 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2508-03936

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Astra : Autonomous Spatial-Temporal Red-teaming for AI Software Assistants Xiangzhe Xu, Guangyu Shen, Zian Su, Siyuan Cheng, Hanxi Guo, Lu Yan, Xuan Chen, Jiasheng Jiang, Xiaolong Jin, Chengpeng Wang, Zhuo Zhang, Xiangyu Zhang Abstract AI coding assistants like GitHub Copilot are rapidly transforming software development, but their safety remains deeply uncertain—especially in high-stakes domains like cybersecurity. Current red-teaming tools often rely on fixed benchmarks or unrealistic prompts, missing many real-world vulnerabilities. We present Astra , an automated agent system designed to systematically uncover safety flaws in AI-driven code generation and security guidance systems. Astra works in three stages: (1) it builds structured domain-specific knowledge graphs that model complex software tasks and known weaknesses; (2) it performs online vulnerability exploration of each target model by adaptively probing both its input space, i.e., the spatial exploration, and its reasoning processes, i.e., the temporal exploration, guided by the knowledge graphs; and (3) it generates high-quality violation-inducing cases to improve model alignment. Unlike prior methods, Astra focuses on realistic inputs—requests that developers might actually ask—and uses both offline abstraction guided domain modeling and online domain knowledge graph adaptation to surface corner-case vulnerabilities. Across two major evaluation domains, Astra finds 11–66% more issues than existing techniques and produces test cases that lead to 17% more effective alignment training, showing its practical value for building safer AI systems. Responsible Red Teaming Statement All simulated attacks, jailbreak prompts, and malicious code examples in this paper were generated and tested in secure, non-production environments. No functioning malware was executed or retained. Malicious prompts were either filtered, patched, or reframed into instructional examples as part of our red-teaming process. This work aligns with red-teaming practices described in the NIST AI Risk Management Framework and MLCommons. Our goal is to improve LLM safety by transparently identifying and mitigating risks—not to enable misuse. 1 Introduction AI enables highly autonomous systems capable of sensing, reasoning, and acting upon their environments, which are rapidly becoming integral to both enterprise operations and consumer-facing services, across numerous domains. In software development, AI such as GitHub Copilot and Amazon Q now assists with tasks like coding and testing, significantly reducing development time and lowering costs. This transformation is reflected by explosive market growth: the global AI-in-software market is projected to grow from USD 160.1 billion in 2023 to over USD 2.5 trillion by 2033 marketus2024aiinsoftware . Despite these trends, significant concerns persist regarding the correctness , security , explainability , and fairness of AI. These properties are essential as errors by AI could lead to errors in code or misaligned behavior in sensitive domains. It is hence critical to ensure AI’s conformance to critical safety properties. The AI assurance market is projected to reach $276 billion by 2030 ft2024assurance , indicating rapidly growing demand. Among the quickly expanding landscape of AI applications, software development assistance stands out as the most widely adopted and commercially successful application. While adoption grows, so does the need for continuous evaluation. To further improve trust and reliability, our goal is to develop automated red-teaming techniques that systematically uncover vulnerabilities in AI’s behavior related to safe coding and software development guidance. A Cognitive Framework for AI Red-Teaming. Motivated by the observation that AI exhibits human-like problem-solving behavior, we adopt a formal framework from cognitive science newell1972human that models human reasoning, in order to analyze existing red-teaming and blue-teaming techniques and to present our own approach. Figure 1: A Cognitive Framework for Red-Teaming: Modeling AI Vulnerabilities through Human Problem-Solving Paradigms As illustrated in Figure 1 , problem-solving is conceptualized as a transformation from an input state (e.g., a user prompt) on the left, which is also called a configuration , to an output state (e.g., the model’s response) on the right. This transformation is governed both by the initial input and the underlying AI model. Safety properties define a designated subspace of acceptable outputs (i.e., the green region inside the output space). Safety violations can arise from two primary sources: inputs that are inherently unsafe, and inputs that are benign but are misprocessed by the model. The former indicates a deficiency in the model’s ability, or a vulnerability , in detecting and preventing malicious intent, while the latter highlights vulnerabilities in the model’s reasoning or decision-making processes when handling otherwise safe inputs. Both fall into the red input region in Figure 1 . Red-teaming aims to discover red regions, whereas blue-teaming aims to remove these regions. We further partition the input space into two subspaces: realistic (the gray half) and unrealistic (the black half), based on whether the input reflects a plausible operational scenario within the model’s intended service domain. For instance, a prompt asking a software development assistant AI to write fiction is considered unrealistic, whereas a request to explain a cross-site scripting vulnerability is realistic. This distinction is essential for understanding why many existing red/blue-teaming techniques succeed or fail—and it plays a central role in the design of our proposed solution. Existing Red-teaming (RT) Techniques. A wide range of red-teaming techniques have been proposed pair ; tap ; pal ; rlbreaker ; deepinception ; autodan ; renellm ; fuzzllm ; codechameleon ; artprompt ; ace ; dra ; simplejailbreak ; pap ; multiverse ; gptfuzzer ; masterkey ; flipattack ; drattack ; cognitiveoverload ; redqueen ; actorattack ; chainofattack ; jigsaw . Many of them target the unrealistic input subspace, leveraging the fact that alignment training for foundation models predominantly focuses on realistic operational contexts, often neglecting atypical or unnatural queries. For example, approaches such as PAP pap , DeepInception deepinception , DRA dra , and AutoDan autodan , uncover vulnerabilities by crafting such adversarial inputs. DeepInspection deepinception constructs a virtual, nested (unrealistic) scene to adaptively evade safety alignment. DRA dra conceals harmful instructions in puzzles irrelevant to malicious tasks. AutoDan autodan automatically generates complex (and in many cases unrealistic) adversarial prompts using a genetic algorithm. Despite their successes, these techniques face several key limitations. First, many of them are spontaneous in nature, relying on creatively constructed, unrealistic scenarios that do not reflect the agent’s actual operational use cases. Whether the model aligns in these edge cases is largely incidental and does not meaningfully inform its behavior in real-world settings. Note that we define the “unrealistic” subspace as prompts that fall outside the system’s intended operational scenarios. Although an adversary could deliberately employ such out-of-domain requests to probe for vulnerabilities, as foundation models continue to advance, particularly in reasoning and alignment, models are increasingly capable of rejecting unrealistic prompts that fall outside their intended service domain. This has been repeatedly observed in our internal blue-teaming efforts and recent tournaments. For example, we found that the latest blue-teaming techniques, including our own, can easily defend the 17 recent red-teaming attacks we have re-implemented pair ; tap ; rlbreaker ; deepinception ; autodan ; renellm ; fuzzllm ; codechameleon ; ace ; dra ; simplejailbreak ; pap ; multiverse ; gptfuzzer ; masterkey ; flipattack ; cognitiveoverload 1 1 1 Used as representative industry references, not as definitive rankings. . Most these attacks work by setting up an unrealistic context for the model (e.g., embedding a request for vulnerable code generation in a fiction). These observations motivate a central design decision in our

Read the complete text