Data Mixing Optimization for Supervised Fine-Tuning of Large Language Models
Yuan Li, Zhengzhong Liu, Eric Xing · 2025
Read Data Mixing Optimization for Supervised Fine-Tuning of Large Language Models by Yuan Li, Zhengzhong Liu, Eric Xing (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-11953.txt.
Introduction
Yuan Li, Zhengzhong Liu, and Eric Xing treated a practical training question as mathematics in 2025: in what proportions should data be mixed when fine-tuning a language model? General-purpose models learn from many domains at once, and the recipe's proportions strongly affect the result. Despite that, data mixing for supervised fine-tuning has been little studied. The authors frame it as an optimization problem aimed at minimizing validation loss.
Their method models how much usable data effectively transfers from each source, borrowing fine-tuning scaling laws. Small experiments with varied mixtures fit the model's parameters, and the fitted model then prescribes optimal weights. Proofs and controlled experiments back the approach. Models trained with the prescribed mixture matched far more exhaustive searches. A principled recipe saves the enormous compute that trial-and-error mixing burns.
Catalog record
| CREATOR | Yuan Li, Zhengzhong Liu, Eric Xing |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 62 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2508-11953
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Data Mixing Optimization for Supervised Fine-Tuning of Large Language Models Yuan Li, Zhengzhong Liu, Eric Xing Abstract Optimizing data mixtures for supervised fine-tuning (SFT) of large language models (LLMs) is critical for developing general-purpose models, yet this area remains underexplored. In this paper, we frame data mixing as an optimization problem and introduce a novel method designed to minimize validation loss. Our approach parametrizes the loss by modeling effective data transferred and leveraging scaling laws for fine-tuning. By experimenting with various small-scale data mixtures, we fit these parameters and derive the optimal weights. We provide both mathematical proofs and empirical results demonstrating that our algorithm achieves excellent overall and individual performance across all domains. Through controlled experiments, we show that models trained with our optimized weights perform on par with those using optimal weights determined via grid search, with per-domain loss only 0.66% higher than the best domain loss from grid search on average. Additionally, we show that reweighting popular SFT datasets using our method improves both validation loss and downstream performance. Finally, we discuss how our method can generalize to guide data selection for domain-specific models and provide insights into SFT. 1 Introduction Large Language Models (LLMs) are general-purpose systems capable of performing a wide range of tasks, such as following instructions and solving mathematical problems. To train these models, a critical factor is to construct training datasets that contain diverse domains (e.g., math, code, healthcare)—a procedure termed data mixing . Data mixing has significant impacts on models’ performance ( Albalak et al. 2023 ; Xie et al. 2024 ) . However, it is impractical to identify the best data mixture by exhaustively trying every possible combination of domain weight. Prior research has concentrated on optimizing data mixtures during the pre-training of LLMs. A prevalent strategy for determining optimal weights of pre-training data involves defining or parametrizing the loss as a function of domain weights and subsequently minimizing this function. This overarching strategy encompasses various approaches. One such approach employs small proxy models to optimize domain weights, which are then applied to train larger models ( Xie et al. 2024 ; Fan et al. 2024 ) . ( Xie et al. 2024 ; Fan et al. 2024 ) . Alternatively, methods like Data Mixing Laws and RegMix ( Ye et al. 2024 ; Liu et al. 2024 ) parameterize the relationship between model performance (e.g., loss) and data mixtures. These methods involve employing a function to estimate the performance and then using the function to optimize the data mixture. Despite significant efforts in pre-training, data mixing for supervised fine-tuning (SFT) remains underexplored. This oversight is unwarranted, as recent research indicates that SFT not only aligns the model stylistically but also infuses knowledge and enhances the model’s capabilities ( Wu et al. 2024 ; Raghavendra et al. 2024 ) . Moreover, existing data mixing methods designed for pre-training are not directly applicable to SFT due to several challenges: 1) Optimizing domain weights using small proxy models may not translate effectively to larger models. Since SFT builds upon models pre-trained with distinct datasets, it is questionable whether the optimal weights derived from proxy models remain valid for the specific large-scale models at SFT stage; 2) Even when the same model is used, parameterizing the loss without accounting for the interplay and scaling effects of different domains can lead to poor extrapolation. At the SFT stage, the goal is to train a general-purpose model with robust performance across multiple domains. Existing loss minimization does not ensure adequate representation for each domain. In fact, these loss parametrizations lead to unbalanced domain weight distributions when scaling up data sizes. These challenges call for a systematic data mixing method for SFT. In this paper, we introduce Data Mixing Optimization , a method that frames data mixing as an optimization problem to determine optimal domain weights for given data budgets. Our approach involves a novel parameterization of validation loss tailored for SFT. This parameterization accounts for both the scaling effects of fine-tuning data and the interactions between different domains. We analyze our optimization method and demonstrate that our method offers two key advantages. First, it shares similar intuition and principles with existing methods ( Xie et al. 2024 ; Fan et al. 2024 ; Kang et al. 2024 ) in up-weighting domains that effectively reduce the loss. Second, it is theoretically robust because we rigorously proved that our algorithm ensures balanced performance across all domains. We conducted extensive experiments across various scenarios, including controlled studies using three distinct domains: instruction following, math, and code; and experiments of re-weighting popular SFT collections. Our results show that our method effectively derives domain weights that minimize validation loss. Furthermore, defining domains as specific tasks (e.g., math, code) can make our method useful for improving downstream performance on specific tasks. We also show that our data mixing optimization can be extended to guide the training of domain-specific LLMs and provide insights into SFT. To summarize, our contributions are three-fold: • Data Mixing Optimization for SFT : We introduce a method that optimizes domain weights for SFT data. Central to this optimization is a novel parameterization of validation loss that models the scaling effects of SFT data and domain interactions. This method is theoretically solid and ensures no domain will underperform. • Extensive Empirical Results on Data Mixing : We present extensive SFT results about data mixing across various LLM classes and model sizes. The findings not only validate the effectiveness of our method but also reveal interesting patterns, such as the similarity in optimal weights among models within the same LLM classes, that pave the way for future research. • Implications and Insights for SFT : We demonstrate that data mixing optimization can guide data selection for specialized tasks and serve as a framework to understand key aspects of SFT, including data scaling and domain interactions. 2 Related Work Due to space limitations, we briefly discuss related work here and leave detailed analysis in Appendix A . Data Mixing. The composition of training data for LLMs has drawn significant interest due to its impact on performance. Data mixing, also known as domain reweighting , is well-studied at the pre-training stage. Methods like DoReMi ( Xie et al. 2024 ) and DoGE ( Fan et al. 2024 ) identify optimal weights using smaller models and subsequently apply these weights to train larger models. Other approaches, such as Data Mixing Laws ( Ye et al. 2024 ) , RegMix ( Liu et al. 2024 ) , and Aioli ( Chen et al. 2024a ) , estimate loss as a function of domain weights and interpolate performance to determine optimal data compositions. However, these methods often struggle to generalize across different model scales or data distributions, limiting their applicability to SFT. To the best of our knowledge, SMART ( Renduchintala et al. 2024 ) is the only method explicitly designed to optimize domain weights for SFT; however, it relies on prompt embeddings and remains model and scale-invariant. In contrast, our proposed approach determines optimal domain weights that depend on both model size and scale, thus avoiding the limitations of previous methods. Supervised Fine-Tuning. SFT, or instruction fine-tuning, involves training pre-trained LLMs with instruction-output