LaMDAgent: An Autonomous Framework for Post-Training Pipeline Optimization via LLM Agents

Taro Yano, Yoichi Ishibashi, Masafumi Oyamada · 2025

Read LaMDAgent: An Autonomous Framework for Post-Training Pipeline Optimization via LLM Agents by Taro Yano, Yoichi Ishibashi, Masafumi Oyamada (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-21963.txt.

Introduction

Taro Yano, Yoichi Ishibashi, and Masafumi Oyamada let language model agents design the whole post-training pipeline for other language models. Their paper appeared in 2025. Post-training, the fine-tuning, preference learning, and merging that tailor a model, is usually hand-built piece by piece. Their framework, LaMDAgent, uses agents to search over model generation tricks, datasets, and settings, guided by task feedback, with little human help.

The agents improved tool-use accuracy by nine points while keeping instruction following intact. They also surfaced strategies human explorers usually overlook. The authors analyze which data and model choices drove the wins. Their claim is that pipeline design itself can be automated, turning a craft bottleneck into a search problem agents can run overnight.

Catalog record

CREATORTaro Yano, Yoichi Ishibashi, Masafumi Oyamada
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE37 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-21963

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

LaMDAgent: An Autonomous Framework for Post-Training Pipeline Optimization via LLM Agents Taro Yano, Yoichi Ishibashi, Masafumi Oyamada Abstract Large Language Models (LLMs) have demonstrated exceptional performance across a wide range of tasks. To further tailor LLMs to specific domains or applications, post-training techniques such as Supervised Fine-Tuning (SFT), Preference Learning, and model merging are commonly employed. While each of these methods has been extensively studied in isolation, the automated construction of complete post-training pipelines remains an underexplored area. Existing approaches typically rely on manual design or focus narrowly on optimizing individual components, such as data ordering or merging strategies. In this work, we introduce LaMDAgent (short for Language Model Developing Agent), a novel framework that autonomously constructs and optimizes full post-training pipelines through the use of LLM-based agents. LaMDAgent systematically explores diverse model generation techniques, datasets, and hyperparameter configurations, leveraging task-based feedback to discover high-performing pipelines with minimal human intervention. Our experiments show that LaMDAgent improves tool-use accuracy by 9.0 points while preserving instruction-following capabilities. Moreover, it uncovers effective post-training strategies that are often overlooked by conventional human-driven exploration. We further analyze the impact of data and model size scaling to reduce computational costs on the exploration, finding that model size scalings introduces new challenges, whereas scaling data size enables cost-effective pipeline discovery. 1 Introduction Large Language Models (LLMs) have undergone rapid development, demonstrating exceptional performance across diverse tasks and significantly impacting both academic and industrial domains, with the rise of high-performing proprietary models ( OpenAI, 2023 ; Anthropic, 2024 ; Google, 2024 ) as well as open-sourced models ( Dubey et al., 2024 ; Yang et al., 2024 ; DeepSeek-AI et al., 2024 ; Abdin et al., 2024 ) . LLM development typically involves pre-training on large-scale web corpora followed by post-training with curated data ( Ouyang et al., 2022 ) , with this study focusing on the latter stage due to the increasing emphasis on post-training driven by the release of models and datasets tailored for domain and task adaptation ( Tie et al., 2025 ) . In post-training, widely adopted approaches include Supervised Fine-Tuning (SFT) using human-created prompt-response pairs and Preference Learning based on preference labels for response pairs ( Rafailov et al., 2023 ; Ethayarajh et al., 2024 ; Song et al., 2024 ; Munos et al., 2024 ) . Furthermore, innovative techniques are rapidly evolving, such as autonomous training data generation and “model merging” that creates new models through arithmetic operations on different model parameters ( Wortsman et al., 2022 ; Ilharco et al., 2023 ; Yadav et al., 2023 ) . To generate superior models, existing studies either manually build pipelines or focus on optimizing specific steps such as fine-tuning data orderings ( Chen et al., 2023 ; Kim and Lee, 2024 ; Pattnaik et al., 2024 ) or model merging strategies ( Ishibashi et al., 2025 ; Akiba et al., 2025 ) . However, full adaptation for target tasks requires combining these methods into integrated pipelines to optimize, yet automating this end-to-end process remains largely unexplored. In this paper, we propose La nguage M odel D eveloping Agent (LaMDAgent), a method that autonomously constructs post-training pipelines using LLM-based agents and continuously improves them based on feedback from the generated model’s performance on target tasks. LaMDAgent treats heterogeneous model improving methods such as supervised fine-tuning, preference learning, or model merging in a unified manner and automates end-to-end post-training pipeline construction by exploring appropriate model generation methods, datasets, hyperparameters, and their optimal application order, thereby reducing the specialized knowledge and human costs required for pipeline construction. Additionally, to reduce computational costs for LaMDAgent’s exploration, we experimentally verify data size scaling and model size scaling, where data size scaling and model size scaling respectively involve exploring pipelines with smaller data quantities and model sizes, then transferring the discovered efficient pipelines to larger data quantities and model sizes. The contributions of this paper are as follows: 1. We propose an LLM Agents-driven framework “LaMDAgent” that autonomously constructs and optimizes post-training pipeline. LaMDAgent treats heterogeneous model improving methods in a unified framework to optimize the entire pipeline in post-training, reducing the specialized knowledge and human costs required for pipeline construction. 2. In our experiments across two distinct settings, we show that LaMDAgent effectively improves mathematical capability by 3.7 points in average accuracy in Experiment 1 and enhances tool utilization accuracy by 9.0 points in Experiment 2 compared to strong baselines, while maintaining general capabilities through the discovery of novel pipelines that are not easily identified by humans. 3. To reduce LaMDAgent’s exploration costs, we verify the effectiveness of data size scaling and model size scaling, finding that model size scalings introduces new challenges, whereas scaling data size enables cost-effective pipeline discovery. 2 Methodology 2.1 Overview We propose a novel method called La nguage M odel D eveloping Agent (LaMDAgent) that fully automates the construction and optimization of language model post-training pipelines using LLM Agents. Figure 1: Overview of our LaMDAgent framework. LaMDAgent first enumerates actions from predefined model improving action types and an object pool containing available data, models, parameters, and other objects (Step1. Action Eunumeration). Next, the agent selects an action based on memory acquired from previous trials and executes the selected action to generate a new model (Step2. Action Selection). Then evaluations on downstream tasks are conducted (Step3. Model Evaluation). Based on the evaluation results of the newly generated model, the agent considers promising future directions and insights, updating the accumulated memory (Step4. Memory Update). Figure 1 illustrates the overview of our proposed method. The proposed method aims to create better models by iteratively repeating the following four steps: 1. Action enumeration , 2. Action selection, 3. Model evaluation, and 4. Memory update. Details are described in the following sections. 2.2 Action Enumeration For simplicity of explanation, we define the following terms: • Object: A concrete entity used in the model training pipeline, such as Llama 3 8B as a model or GSM8k as training data. • Action: An action is a model improving method that takes multiple objects, including models, as input and outputs a new model. An action is defined by an action type such as "SFT" and the objects used, such as specific data, models, or hyperparameters. We obtain possible actions by enumerating all combinations of action types and objects. Specifically, we use predefined action types and objects that include both pre-prepared datasets and models, as well as models and data obtained during the iteration. For example, if we have the action type "SFT" is defined to take (base model, training data) as input objects, and we have Gemma2 2B as a base model and GSM8k and MATH as training data, then possible actions can be enumerated as (Gemma2 2B, GSM8k) and (Gemma2 2B, MATH). 2.3 Action Selection We use the agent to select one promising model improvement action from possible actions. During action selection, we provide the agent with a prompt for action selection and parse its output to determine the action. In practice, rather than providing all action candidates and having the agent output a single action index, we first have the agent select an action type in one inference step and identify the required object types based on the action

Read the complete text