FABRIC: F ramework for A gent- B ased R ealistic I ntelligence C reation Weaving Synthetic Enterprise Data for Training Autonomous Agents
Abhigya Verma, Seganrasan Subramanian, Nandhakumar Kandasamy, Naman Gupta · 2025
Read FABRIC: F ramework for A gent- B ased R ealistic I ntelligence C reation Weaving Synthetic Enterprise Data for Training Autonomous Agents by Abhigya Verma, Seganrasan Subramanian, Nandhakumar Kandasamy, Naman Gupta (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-17995.txt.
Introduction
Abhigya Verma and three co-authors synthesize training records for software agents without any human labelers. Agents need data that link a user's goal to tool details, well-formed calls, and proof that the calls ran. Gathering that from people is slow, costly, and hard to scale, so the team generates it with language models alone.
Their FABRIC frame splits the job into small pipelines that emit whole records: task, tools, policy sketch, chat, and run traces. Strict rules keep each record machine-readable and aligned across its parts. Builders of agent systems get a recipe for practice data that can be checked, not just read.
Catalog record
| CREATOR | Abhigya Verma, Seganrasan Subramanian, Nandhakumar Kandasamy, Naman Gupta |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 139 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2510-17995
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
FABRIC: F ramework for A gent- B ased R ealistic I ntelligence C reation Weaving Synthetic Enterprise Data for Training Autonomous Agents Abhigya Verma, Seganrasan Subramanian, Nandhakumar Kandasamy, Naman Gupta Abstract Large language models (LLMs) are increasingly deployed as agents , expected to decompose goals, invoke tools, and verify results in dynamic environments. Realizing these capabilities requires access to agentic data [ 1 ] —structured interaction records that couple user intents with tool specifications, argument-grounded calls, and verifiable execution traces. However, collecting such data from human annotators is costly, time-consuming, and difficult to scale. We present a unified framework for synthesizing agentic data using only LLMs, without any human-in-the-loop supervision. This framework decomposes generation into modular pipelines that produce complete interaction records spanning task specifications, tool definitions, policy pseudocode, natural language exchanges, and execution traces. Records conform to strict syntactic and semantic constraints, ensuring machine-parseability and faithful alignment across inputs, outputs, and tool calls. Beyond single tasks, there is support for both multi-task and multi-turn agent interactions, enabling the construction of datasets that reflect the full spectrum of tool-use competencies. To ensure quality and consistency, the framework integrates constrained generation formats, JSON-schema validation, and judge-based filtering. This paper formalizes the schema for agentic records, details the prompt design principles that guide generation, and introduces scalable pipelines for high-quality synthetic data. By providing a reproducible, LLM-only alternative to manual collection, hence advancing the development of agentic LLMs capable of robust tool use. K eywords Agentic data \cdot Synthetic data \cdot Large language models \cdot Function calling \cdot Tool use \cdot Execution traces \cdot Multi-turn dialogue \cdot BFCL \cdot Pseudo code \cdot JSON schema 1 Introduction Large language models (LLMs) are increasingly deployed as agents that decompose goals, invoke external tools, and coordinate multi-step workflows [ 2 ] . Realizing these capabilities requires access to agentic data —structured, machine-interpretable records that capture the full trajectory of tool-augmented task execution, including user intent, tool specifications, decision logic, and verifiable execution traces. 1.1 Motivation Agentic data differs fundamentally from conventional text corpora. Whereas standard datasets emphasize linguistic fidelity, agentic datasets must encode executable actions. Each record specifies not only natural language exchanges but also function calls, argument bindings, and execution outcomes. Such structured supervision is essential for enabling LLMs to learn robust tool-usage policies and for evaluating them on correctness, alignment, and reproducibility. 1.2 Challenges in Data Acquisition Manual collection of agentic data at scale is infeasible. Human annotators must understand APIs, generate type-correct arguments, and validate tool responses across domains, a process that is costly, error-prone, and difficult to scale. Surveys of synthetic data generation [ 3 , 4 , 5 ] highlight the benefits of automatic pipelines: scalability, diversity, and cost reduction. However, agentic settings impose stricter requirements due to the complexity of reasoning, planning, and execution fidelity [ 6 , 7 ] . 1.3 Requirements for Synthetic Agentic Data From prior work and system analyses, we identify the following requirements for automatically generated agentic data: • Logical consistency : argument values must respect type and semantic constraints, and call sequences must preserve dependency structure. • Execution fidelity : traces must be verifiable and correspond to replayable executions. • Schema validity : outputs must conform to machine-parseable schemas with strict I/O alignment. • Diversity and coverage : datasets should include varied workflows, counterfactuals, and edge cases to improve robustness. • Scalability through automation : reliance on human-in-the-loop supervision must be eliminated to achieve sufficient scale. 1.4 Research Questions This work is guided by three research questions: 1. How can an LLM-only framework generate synthetic agentic data that meets structural and semantic requirements without human supervision? 2. Which modular pipelines best span the granularity of agentic supervision, from atomic function calls to long-horizon, multi-turn tasks? 3. How can automated validation and schema enforcement guarantee that generated records are directly usable for training and evaluation? We propose a unified framework SYTHIA: S ynthetic Y ielding of T raining data for H uman- I ndependent A gents . The framework consists of four modular pipelines, each designed to generate a distinct class of agentic data. 2 Related Work Recent advances in large language model (LLM) agents have spurred research across four interrelated areas: (1) simulation environments for realistic agentic behavior, (2) benchmarks for tool-augmented reasoning and evaluation, (3) methods for constructing synthetic agentic datasets, and (4) frameworks for tool-use, function-calling, and structured execution traces. We review representative efforts in each category and situate our framework relative to these lines of work. Name Year Description Self-Instruct [ 8 ] 2022 LLM-only instruction synthesis; seed for fully synthetic data pipelines. Mind2Web [ 9 ] 2023 Human-annotated user trajectories over real websites for generalist agent training. AgentBench [ 10 ] 2023 Multi-domain agentic benchmark across reasoning and interaction settings. AutoGen [ 11 ] 2023 Multi-agent conversational framework for synthetic data generation. CAMEL [ 12 ] 2023 Communicative self-play agents for role-driven synthetic data. ToolCoder [ 13 ] 2023 Teaches code generation models to use API tools with schema alignment. TaoBench [ 14 ] 2023 Task-oriented benchmark for evaluating LLM agents on multi-step tool-use and reasoning tasks. Magpie [ 15 ] 2024 Alignment data generation from scratch with minimal inputs. UltraFeedback [ 16 ] 2024 Large-scale preference data via AI feedback loops. Arena Learning [ 17 ] 2024 Iterative post-training data flywheel based on simulated chatbot arenas. 1B Personas [ 18 ] 2024 Large-scale persona-driven synthetic data generation. LLM2LLM [ 19 ] 2024 Iterative refinement of data via LLM-to-LLM enhancement. WebArena [ 20 ] 2024 Realistic web environment with functional sites and long-horizon agent tasks. BrowserGym [ 21 ] 2024 Gym-style framework and benchmark suite for web-based agent training. AppWorld [ 22 ] 2024 Multi-app environment for interactive code-based agent workflows. WorkArena [ 23 ] 2024 Evaluation framework on production websites for task-solving agents. ST-WebAgentBenchh [ 24 ] 2024 Evaluates web agent safety and trustworthiness in open-ended settings. AgentInstruct [ 25 ] 2024 Agentic flows for generative teaching and data creation. AgentTrek [ 26 ] 2025 Converts web tutorials into executable agent trajectories. BFCL [ 27 ] 2025 Leaderboard for structured function-calling accuracy and argument correctness. Table 1: Representative works across environments, benchmarks, synthetic data generation, and structured tool-use. 2.1 Web-Based and Interactive Agent Environments Simulation environments provide reproducible testbeds for evaluating autonomous agents. WebArena [ 20 ] , BrowserGym [ 21 ] , and WorkArena [ 23 ] target realistic web tasks, offering functional sites and long-horizon goals. AppWorld [ 22 ] generalizes this paradigm to multi-app ecosystems with user agents, supporting code-rich workflows. These environments support controlled evaluation but remain limited by dependence on pre-built platforms and runtime execution. 2.2 Benchmarks for Tool-Augmented Agents Benchmarks assess the reasoning, tool-use, and safety of language agents. AgentBench [ 10 ] measures multi-domain decision-making abilities across interactive tasks. The Berkeley Function-Calling Leaderboard (BFCL) [ 27 ] isolates structured function-calling accuracy, focusing on correctness of names, arguments, and outputs. Safety-oriented benchmarks such as ST-WebAgentBench [ 24 ] probe robustness under adversarial or policy-sensitive settings. Recent extensions like BenchAgents [ 28 ] and Benchmark Self-Evolving [ 29 ] explore automated benchmark generation through agent interactions, complementing our focus on automated data synthesis. 2.3 Agentic Dataset Construction and Synthetic Data Generation Training effective