Safactory: A Scalable Agentic Infrastructure for Training Trustworthy Autonomous Intelligence

Shanghai AI Laboratory ∗ · 2026

Read Safactory: A Scalable Agentic Infrastructure for Training Trustworthy Autonomous Intelligence by Shanghai AI Laboratory ∗ (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-06230.txt.

Introduction

Xinquan Chen and forty co-authors present Safactory, a combined infrastructure for training trustworthy autonomous agents. As models move from chat assistants to agents that act over long horizons with tools, risks surface through real environment interaction. Current support systems stay split across testing, data handling, and agent improvement, so risks are hard to find and fix in a steady loop.

Safactory joins three platforms into one evolutionary pipeline. A parallel simulation platform generates agent runs at scale. A trustworthy data platform stores those runs and extracts reusable experience from them. An autonomous evolution platform then improves models through asynchronous reinforcement learning and on policy training. The authors describe the system as the first unified pipeline of its kind for next generation agent training.

Catalog record

CREATORShanghai AI Laboratory ∗
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE97 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.06230

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Safactory: A Scalable Agentic Infrastructure for Training Trustworthy Autonomous Intelligence Shanghai AI Laboratory ∗ Abstract As large models evolve from conversational assistants into autonomous agents, challenges increasingly arise from long-horizon decision making, tool use, and real environment interaction. Existing agenticinfrastructure remain fragmented across evaluation, data management, and agent evolution, making it difficult to discover risks systematically and improve models in a continuous closed loop. In this report, we present Safactory , a scalable agent factory for trustworthy autonomous intelligence. Safactory integrates three tightly coupled platforms: a Parallel Simulation Platform for trajectory generation, a Trustworthy Data Platform for trajectory storage and experience extraction, and an Autonomous Evolution Platform for asynchronous reinforcement learning and on-policy distillation. As far as we know, Safactory 1 1 1 https://github.com/AI45Lab/Safactory is the first framework to propose a unified evolutionary pipeline for next-generation trustworthy autonomous intelligence. \keepXColumns Figure 1 : Safactory platform overview diagram. † † footnotetext: ∗ Please cite this work as “Shanghai AI Lab (2025)”. Full authorship contribution statements appear at the end of the report. Correspondence regarding this technical report can be sent to [email protected] . 1 Introduction 1.1 A Panoramic View of AI Agent Safety Risks With the rapid development of applications such as companion AI, DeepResearch, intelligent terminals, and embodied AI, large language models are moving from ”conversational assistants” to ”autonomous agents.” These systems are not only responsible for answering questions but also continuously perceive their environment, plan tasks, invoke tools, and directly influence external systems. This brings stronger application capabilities, but also imposes higher safety requirements. Unlike traditional single-turn models, the risks of agents are not limited to harmful outputs; they accumulate progressively throughout the process of ”perception → decision-making → execution → feedback,” ultimately manifesting as real-world environmental consequences. As shown in Figure 2 , violations, jailbreak inductions, and explicit attacks are merely the risks above the surface. Beneath the waterline lie more insidious issues such as tool misuse, state drift, process loss of control, data contamination, cross-turn inducement, and cascading errors. These risks are often harder to detect and more difficult to cover with static testing. Such risks typically exhibit three characteristics. First, they often emerge in long-horizon, multi-step interactions rather than single-turn input-output pairs. Second, they are closely tied to environmental states, tool interfaces, and execution contexts, making them difficult to characterize using static benchmarks alone. Third, when they do occur, the consequences often go beyond degraded response quality — they may further lead to erroneous operations, privacy breaches, system damage, or even real-world losses. Therefore, trustworthy AI for high-value scenarios requires not just ”safer responses,” but more importantly, the ability to remain controllable, auditable, and recoverable while taking continuous action. Figure 2 : The Iceberg Model of AI Safety Risks. 1.2 Limitations of Existing Systems To address the above challenges, existing AI agent safety systems still exhibit significant gaps across three foundational dimensions: evaluation, data, and evolution. These issues are not isolated from one another. Collectively, they result in safety efforts often remaining in a passive ”find a problem, fix a problem” mode, making it difficult to form a continuous closed loop oriented toward real-world deployment scenarios. Limitations of Evaluation Infrastructure Existing security evaluations are still dominated by static question-answering, single-round adversarial examples, or fixed task sets. Such approaches are suitable for answering whether a model ”will say something wrong,” but struggle to answer whether an agent ”will do something wrong during continuous execution.” When risks depend on tool invocation chains, changes in environmental states, and the accumulation of long-horizon decisions, single-turn interactive evaluations can only capture the surface of the iceberg, making it difficult to systematically expose deep failure modes. Even in executable environments, many existing frameworks remain centered around fixed tasks and offline logs, lacking capabilities such as high-concurrency retesting, dynamic perturbation injection, and joint analysis of process–consequence coupling. For the relevant technical landscape, see Section 2.1 . Limitations of Data Infrastructure Currently, many evaluation and training frameworks treat data merely as a byproduct of the runtime process, with common practices relying on in-memory caches and temporary local files for storage. This approach struggles to stably support large-scale heterogeneous data management, nor does it facilitate cross-task, cross-version, or cross-environment retrieval, reuse, and auditing. More importantly, agents continuously generate a large volume of trajectories, reward signals, and failure contexts during evaluation, execution, and training. These are essentially high-value data assets, yet existing systems generally lack a unified mechanism for their structured accumulation, automated analysis, and subsequent feedback into model optimization. For further discussion, see Section 2.2 . Limitations of Evolution infrastructure At the level of model evolution, training, evaluation, and deployment remain largely disconnected from one another. Issues identified during evaluation are difficult to promptly convert into training samples, and real-world feedback generated during execution is also hard to smoothly reintegrate into subsequent optimization workflows. Meanwhile, reinforcement learning and online distillation for agent scenarios are transitioning from standalone algorithm implementations to system-level orchestration problems. However, most existing training frameworks are still designed around local coordination between rollout and trainer components, and are insufficient to stably support complex training closed loops that involve multi-environment concurrency, trajectory staleness control, asynchronous sampling, and online policy updates. For a comprehensive review, see Section 2.3 . 1.3 Safactory: Building a Complete Closed Loop for Trustworthy AI To address the above limitations, Safactory proposes an infrastructure framework for trustworthy autonomous agents. Centered around three core platforms — a Parallel Execution Middle Platform, a Trustworthy Data Middle Platform, and an Autonomous Evolution Middle Platform — it establishes a complete closed loop that integrates risk discovery, data accumulation, and capability repair and reinforcement. Overall, as shown in Figure 1 , Safactory does not treat security evaluation as a one-time tool-based process, but rather organizes it as a data and training system that can run continuously, be iteratively optimized, and be repeatedly reused. Among these, the Parallel Execution Middle Platform targets executable sandboxes for Android, computer environments, and embodied scenarios. It supports environment pool preheating, asynchronous scheduling, and concurrent re-evaluation of multiple trajectories, as well as the injection of attacks and perturbations during execution, enabling more systematic discovery of safety risks in long-horizon interactions. The Trustworthy Data Middle Platform is responsible for consolidating the multimodal data, step-level trajectories, and behavioral signals generated during evaluation and training into retrievable, analyzable, and auditable data assets, and performs safety auditing, sample filtering, and experience distillation through an intent-driven data engine. The Autonomous Evolution Middle Platform further integrates these data assets into asynchronous reinforcement learning and online policy optimization workflows, continuously generating trainable samples under version control, trajectory filtering, and group consistency constraints, thereby supporting the simultaneous improvement of model capabilities and safety. Therefore, Safactory’s goal is neither to improve a single evaluation metric in isolation, nor to merely provide point solutions such as a data tool or a training framework. What it aims to establish is a closed-loop pathway of ”discovering risks through execution, consolidating evidence through data, completing repairs through evolution, and verifying effectiveness through re-execution.” Along this path, safety is no longer a one-time check conducted before model deployment, but rather becomes an infrastructure capability throughout the entire lifecycle of an agent. 2 Related Work 2.1 Agent Evaluation Framework As model capabilities and deployment scenarios continue

Read the complete text