AI Scientist via Synthetic Task Scaling
Ziyang Cai, Harkirat Behl · 2026
Read AI Scientist via Synthetic Task Scaling by Ziyang Cai, Harkirat Behl (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-17216.txt.
Introduction
Cai, Saeidi, and Behl present ML-AutoResearch, a pipeline that builds training tasks for machine learning research agents on its own. Training such agents needs process level supervision, and hand collected data are costly. Their tasks define a full research cycle: problem setup, dataset choice, baseline build, and step by step improvement. Tasks are grounded in real datasets and refined by self debugging. Agents trained on the synthetic trajectories gained across three benchmarks and two model families.
Catalog record
| CREATOR | Ziyang Cai, Harkirat Behl |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 62843 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.17216
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
AI Scientist via Synthetic Task Scaling Ziyang Cai, Harkirat Behl Abstract With the advent of AI agents, automatic scientific discovery has become a tenable goal. Many recent works scaffold agentic systems that can perform machine learning research, but don’t offer a principled way to train such agents—and current LLMs often generate plausible-looking but ineffective ideas. To make progress on training agents that can learn from doing, we provide a novel synthetic environment generation pipeline targeting machine learning agents. Our pipeline automatically synthesizes machine learning challenges compatible with the SWE-agent 15 framework, covering topic sampling, dataset proposal, and code generation. The resulting synthetic tasks are 1) grounded in real machine learning datasets, because the proposed datasets are verified again the Huggingface API and are 2) verified for higher quality with a self-debugging loop. To validate the effectiveness of our synthetic tasks, we tackle MLGym 6 , a benchmark for machine learning tasks. From the synthetic tasks, we sample trajectories from a teacher model (GPT-5 12 ), then use the trajectories to train a student model (Qwen3-4B and Qwen3-8B 14 ). The student models trained with our synthetic tasks achieve improved performance on MLGym rasing the AUP metric by 9% for Qwen3-4B and and 12% for Qwen3-8B. 1 Introduction One of the key goals of AI is to autonomously perform scientific discovery—formulating hypotheses, design and conduct experiments, analyze results, and integrate new knowledge. Recent systems such as AI Scientist 5 , Co-Scientist 3 , and AlphaEvolve 7 show that AI can already carry out basic research and algorithmic improvement. Meanwhile, large language models (LLMs) have acquired extensive knowledge of machine learning theory, literature, and coding patterns. Yet, knowledge alone is not enough: to convert understanding into effective research, AI agents must gain experience in executing multi-step, goal-directed tasks. Existing research agents are often trained only on final outputs—papers, code, or datasets—ignoring the iterative processes that lead to discoveries, such as debugging, experimental failures, and step-by-step reasoning. To address this, we focus on end-to-end machine learning research task, and introduce a scalable pipeline for synthetic ML task generation that produces rich, agentic trajectories with minimal manual effort. Critically, this pipeline is compatible with the task-agnostic SWE-Agent framework, enabling models to learn from a wide variety of ML tasks across domains. By fine-tuning on these trajectories, agents gain structured experience in the full research cycle, from hypothesis to evaluation. We use our method to tackle MLGym 6 , a benchmark for machine learning agents. MLGym includes 13 machine learning tasks of various complexity. The goal of the agent is to improve upon a baseline implementation, and produce an implementation that achieves a better final score. The score is a scalar, and may vary from task to task, and usually corresponds to training accuracy, loss, win rate, etc. Based on SWE-agent framework, there is a set number of 50 rounds, and each round, the agent produce a "rational" and an "action", which may include browsing files, editing code, running commands, and submitting its final implementation. Multiple submission are allowed, which reflects iterative optimization process of the final score. Our environment synthesis system produces around 500 tasks, which results in a dataset of around 30k agent trajectories. Training Qwen3-4B and 8B models 14 on these trajectories show performance gain, increasing performance on most individual tasks in the benchmark and increase performance of Qwen3-4B and Qwen3-8B by 9% and 12% respectively. By combining broad knowledge, large-scale agentic experience, and task-agnostic training, our approach provides a practical path toward AI systems capable of autonomous, iterative scientific discovery. Figure 1: Illustration of our task and trajectory generation workflow. Crucially, the task generation process does not require human supervision. Instead, it automatically samples machine learning topics and proposes dataset to use in the task. To resolve compilation issues in generated tasks, we further enhance the generation with a debug loop instead of immediately discarding the task altogether. 2 Methodology To advance of frontier of ML agents, we scale up automatic agent task synthesis. Since we target ML capabilities, we aim to synthesize many tasks for Machine Learning. Then, a teacher model would generate trajectories, based on synthetic tasks, which becomes viable training data for downstream models. 2.1 Phase 1: Environment Synthesis The main driver of our method is synthetic environment generation of ML tasks. We use a multistage environment generation pipeline that focus on task diversity and task validity: 1. Topic Sampling Sample n distinct machine learning topics from the model. 2. Task and dataset proposal For each topic, the teacher model generates a task description and propose a HuggingFace dataset to use. We use the HuggingFace search API to find the closest match with the model’s proposal. We allow tasks that has no dataset (for example game theoretic tasks). If there is a match, then we enrich the dataset description with examples of the dataset rows fetched from Huggingface. If there is no match, the task is discarded. 3. Config and starter code generation From the task and dataset descriptions, we generate task and dataset config files compatible with the MLGym execution environment. We also generate all the starter code files for the task as well as any extra helper code. In the end, we will have baseline implementation and an evaluation file. Figure 2: Generated trajectory count for each task. We select 20 generated tasks and show the number of successful trajectories for each task. Because of the unsupervised nature of our pipeline, we don’t expect all tasks to successfully create all 256 trajectories. 2.2 Phase 2: Environment Verification Since each step of the pipeline may be prone to error, we need to verify validity of the tasks as best as we can. To do this, we plug the new task into MLGym, and run the task using a GPT-5 agent to obtain the baseline performance and at least one agent trajectory. If there is an error during the execution, we collect the errors and feed them back to the model in step 3 (starter code generation) with probability p_{\text{debug}} or restart from step 3 with probability 1-p_{\text{debug}} . The iterative debug process can continue at most k times. If the task still fails after maximum iterations, we discard the task. Crucially, this environment synthesis pipeline requires no human input, and is highly scalable through parallel compute. 2.3 Phase 3: Trajectory Generation & Filtering Large scale sampling To sample a large amount of agent trajectories for training, we run the synthetic tasks in parallel in a HPC cluster. Each task occupies one GPU, and we aim to collect 256 trajectories per tasks. Even though the tasks are validated, they can still fail in many ways. The cluster environment further impacts trajectory generation through file system and containerization instabilities. Figure 2 qualitatively show the diversity of our generated tasks. Trajectory filtering The collected trajectories are further filtered based on agent performance. Right now, we simply choose the trajectories where the agent completes at least one successful submission. This filter is sufficient for many pathological cases where the agent is stuck in debugging loops. We also filter the trajectories based on length, rejecting any trajectories over 48K tokens long. During training, we further truncate the trajectories to 32K tokens. 3