Subset Selection for Fine-Tuning: A Utility-Diversity Balanced Approach for Mathematical Domain Adaptation

Madhav Kotecha, Vijendra Kumar Vaishya, Smita Gautam, Suraj Racha · 2025

Read Subset Selection for Fine-Tuning: A Utility-Diversity Balanced Approach for Mathematical Domain Adaptation by Madhav Kotecha, Vijendra Kumar Vaishya, Smita Gautam, Suraj Racha (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-01523.txt.

Introduction

Madhav Kotecha, Vijendra Kumar Vaishya, Smita Gautam, and Suraj Racha pick fine tuning subsets for math domain adaptation by balancing utility and diversity. Full set training is costly, so a smart pick can save time and compute. Their work came out in twenty twenty-five.

Utility scores blend perplexity with chain of thought loss to flag hard, high yield cases. Diversity scores keep the pick spread across math subfields. The aim is near full set skill from a small, well chosen slice. Tests on two model families pit the method against random picks, diversity only picks, and top past subset methods.

Catalog record

CREATORMadhav Kotecha, Vijendra Kumar Vaishya, Smita Gautam, Suraj Racha
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE22 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-01523

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Subset Selection for Fine-Tuning: A Utility-Diversity Balanced Approach for Mathematical Domain Adaptation Madhav Kotecha, Vijendra Kumar Vaishya, Smita Gautam, Suraj Racha Abstract We propose a refined approach to efficiently fine-tune large language models (LLMs) on specific domains like the mathematical domain by employing a budgeted subset selection method. Our approach combines utility and diversity metrics to select the most informative and representative training examples. The final goal is to achieve near-full dataset performance with meticulously selected datapoints from the entire dataset while significantly reducing computational cost and training time and achieving competitive performance as the full dataset. The utility metric incorporates both perplexity and Chain-of-Thought (CoT) loss to identify challenging examples that contribute most to model learning, while the diversity metric ensures broad coverage across mathematical subdomains. We evaluate our method on LLaMA-3 8B and Phi-3 models, comparing against several baseline approaches, including random selection, diversity-based sampling, and existing state-of-the-art subset selection techniques. 1 Introduction Fine-tuning large language models (LLMs) on domain-specific tasks often requires substantial computational resources and large datasets. This is particularly intriguing in the mathematical domain, where problems vary widely in difficulty and structure. Current approaches either use the entire dataset (which is highly expensive in terms of computational expense) or select random subsets (potentially missing the important examples that have the highest gain). We need the novel diversity-based subset selection that ensures that the subset selected is inclusive and representative of the entire large dataset. Diversity-based sampling methods such as the Determinant Point Process (DPP) help reduce redundancy but may not prioritize difficult or informative examples. The proposed approach addresses these limitations by developing a selection strategy that balances both the informativeness and diversity of training examples, enabling more efficient fine-tuning while maintaining performance. The requirement of judicially selecting each data point arises as we progress toward the goal of subset selection, as every inclusion of a data point further adds up to the final gain we get by the particular generated subset. By applying this informed subset selection, we aim to achieve near-full dataset performance while reducing fine-tuning cost and carbon footprint. The mathematics domain presents a particularly interesting test case due to its hierarchical nature, where understanding of fundamental concepts is often prerequisite to solving more complex problems and correctly learning the reasoning of one mathematical equation allows similar to be computed accordingly, the main approach that allows the data pruning of the entire dataset. 2 Background and Motivation The rapid advancement of LLMs demands the optimized solution for the existing problem that eventually decrease resources like time and cost. However, fine-tuning these models on domain-specific tasks presents significant challenges. The DELIFT Agarwal et al. (2024) proposed solution designed to address data-selection across all stages of fine-tuning by the help of a single framework. It uses the intuition of in-context examples to get the information gain of each data point relative to the current state of the model. DELIFT Agarwal et al. (2024) combines the pairwise utility with submodularity as proposed by SMART Renduchintala et al. (2024) . The approach ensures that it selects a diverse, non-redundant subset uniquely curated for each phase of fine-tuning. The main challenge that this study proposes to solve is as below: • Computational Cost: Fine-tuning small models and infer for the large model based on that is proposed by Small2Large Yang et al. (2024) . It helps to reduce computation significantly by eliminating the requirement of performing the experiment on a large model to get the evaluation done for the selected method. • Dataset Size: Mathematical datasets can be large and diverse, covering various topics and difficulty levels. The selection of data-points by consideration of the budget is proposed by Banerjee and Chakraborty (2020) that also works under significant resource constraints • Example Importance: Not all training examples contribute equally to model performance, with some being more informative than others. Prior work has shown that the some of the easiest examples from the dataset have no real contributions; eliminating some of the easiest examples can be more influential in teaching mathematical reasoning patterns. • Domain Complexity: Mathematics encompasses numerous subdomains with different reasoning patterns and solution strategies, making it challenging to ensure comprehensive coverage with a small subset. The variant of the submodular function proposed by SMART Renduchintala et al. (2024) allows us to eliminate the entire cluster of the example at once, which can be easily handle by some of the data-points of another nearby cluster. • Transfer Learning Efficiency: Efficient transfer learning requires identifying examples that address gaps in the model’s current knowledge rather than reinforcing already learned patterns. Recent research in efficient fine-tuning has explored various approaches to address these challenges. Random subset selection, while computationally efficient, often results in suboptimal performance due to missing critical examples. Diversity-based methods like DPP ensure broad coverage but may include easy examples that contribute little to model improvement. Utility-based approaches select challenging examples but may overemphasize similar problem types, leading to poor generalization. Our work is motivated by the observation that an ideal subset should balance both utility (selecting informative examples) and diversity (ensuring broad domain coverage). This is particularly important in mathematics, where understanding diverse problem-solving strategies is crucial for developing robust reasoning capabilities. 3 Related Work 3.1 Efficient Fine-tuning Approaches Parameter-efficient fine-tuning methods like LoRA (Low-Rank Adaptation) Hu et al. (2022) that focus on a small subset of trainable parameters instead of the entire model; similarly, prefix tuning has gained popularity for reducing memory requirements. However, these approaches still process the entire dataset, limiting computational savings. The proposed study is complementary to these methods, as our subset selection can be combined with parameter-efficient techniques for maximum efficiency. 3.2 Data Selection for Fine-tuning Several approaches have been proposed for selecting subsets for fine-tuning: • Uncertainty-based Selection: Methods like active learning select examples where the model has high uncertainty, but these can be computationally expensive to identify in large datasets. As the dataset size increases, the computation required to select highly uncertain examples also increases. • Diversity-based Selection: DPP and k-means clustering select diverse examples but may not prioritize informativeness, as diversity does not always focus on improving the model on highly challenging tasks. • Hybrid Approaches: Recent work by Yang et al. (2024) proposed SmallToLarge (S2L), which uses a smaller model’s training trajectory to identify valuable examples for larger models. The fine-tuning on smaller model is helpful to get the expected results and report for larger models. • Submodular Optimization: Renduchintala et al. (2024) introduced SMART, a submodular data mixture strategy for instruction tuning that balances diversity and utility by considering that the gain by adding the element in the subset is always greater than or equal to the gain we get by adding the element in its superset. Our approach builds upon these methods but is specifically tailored to the specialized domains like mathematics, incorporating domain-specific utility metrics like Chain-of-Thought loss that capture reasoning complexity. 4 Proposed Approach We studied the subset selection method that uses utility value as the key metric to select the most valuable training examples out of all the examples present in the dataset. The implementation of a unified and computationally efficient data selection framework that works with multiple stages of fine-tuning

Read the complete text