D 3 : Diversity, Difficulty, and Dependability-Aware Data Selection for Sample-Efficient LLM Instruction Tuning

Jia Zhang, Chen-Xi Zhang, Yao Liu, Yi-Xuan Jin, Xiao-Wen Yang, Bo Zheng, Yi Liu, Lan-Zhe Guo · 2025

Read D 3 : Diversity, Difficulty, and Dependability-Aware Data Selection for Sample-Efficient LLM Instruction Tuning by Jia Zhang, Chen-Xi Zhang, Yao Liu, Yi-Xuan Jin, Xiao-Wen Yang, Bo Zheng, Yi Liu, Lan-Zhe Guo (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-11441.txt.

Introduction

Eight researchers present a method for picking the best training examples for language models, in a 2025 paper. Small, well-chosen data sets can teach models to follow instructions better than huge, messy ones. The hard part is finding the gold inside a large pool without checking it all by hand. The team's method, called D3, scores examples on diversity, difficulty, and dependability.

A scoring step rates how distinct each example is, how hard it is to predict, and how trustworthy an outside model judges it. A selection step then solves for the subset that best balances all three traits, and the steps can repeat with feedback. In tests on public data and a live shopping broadcast app, models trained on under a tenth of the data matched or beat full-data models. The authors pitch the method as a way to cut training cost.

Catalog record

CREATORJia Zhang, Chen-Xi Zhang, Yao Liu, Yi-Xuan Jin, Xiao-Wen Yang, Bo Zheng, Yi Liu, Lan-Zhe Guo
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE49 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2503-11441

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

D 3 : Diversity, Difficulty, and Dependability-Aware Data Selection for Sample-Efficient LLM Instruction Tuning Jia Zhang, Chen-Xi Zhang, Yao Liu, Yi-Xuan Jin, Xiao-Wen Yang, Bo Zheng, Yi Liu, Lan-Zhe Guo Abstract Recent advancements in instruction tuning for large language models (LLMs) suggest that a small, high-quality dataset can significantly equip LLMs with instruction-following capabilities, outperforming large datasets often burdened by quality and redundancy issues. However, the challenge lies in automatically identifying valuable subsets from large datasets to boost both the effectiveness and efficiency of instruction tuning. In this paper, we first establish data selection criteria based on three distinct aspects of data value: diversity, difficulty, and dependability, and then propose the D 3 method comprising two key steps of scoring and selection. Specifically, in the scoring step, we define the diversity function to measure sample distinctiveness and introduce the uncertainty-based prediction difficulty to evaluate sample difficulty by mitigating the interference of context-oriented generation diversity. Additionally, we integrate an external LLM for dependability assessment. In the selection step, we formulate the D 3 weighted coreset objective, which jointly optimizes three aspects of data value to solve for the most valuable subset. The two steps of D 3 can iterate multiple rounds, incorporating feedback to refine the selection focus adaptively. Experiments on both public datasets and the real-world Taobao Live application demonstrate the effectiveness of D 3 in endowing LLMs with competitive or even superior instruction-following capabilities using less than 10% of the entire dataset. 1 Introduction With the success of large language models (LLMs) Brown et al. 2020 ; Touvron et al. 2023 , the application of data-driven AI has been significantly advanced across various industries and scenarios. In the LLM paradigm, large-scale pre-training on massive corpora equips models with knowledge and text generation capabilities based on next-token prediction Naveed et al. 2024 . Moreover, several post-training methods and techniques Bai et al. 2022 ; Wang et al. 2024b ; Rafailov et al. 2024 could further enhance the model’s alignment and performance on downstream tasks. In particular, instruction tuning Ouyang et al. 2022 ; Taori et al. 2023 , which fine-tunes models with labeled pairs of instruction and response, plays a crucial role in enabling LLMs’ instruction-following capabilities across various tasks. Early studies on instruction tuning concentrated on creating extensive and cross-task datasets to enhance the ability of LLMs to follow instructions at a significant cost Wang et al. 2022 ; Xu et al. 2023 ; Wang et al. 2023 . However, recent work Zhou et al. 2023 has shown that a small, carefully curated set of high-quality data can effectively equip LLMs with instruction-following capabilities instead of relying on large datasets. This finding has prompted initial efforts in automated data selection Xia et al. 2024 ; Cao et al. 2024 ; Du et al. 2023 ; Li et al. 2024a ; Li et al. 2024b , where researchers attempt to automatically identify the valuable subset of instruction data for fine-tuning. Despite their potential, these methods generally face three limitations: 1) Many approaches rely on a single metric for data selection, neglecting comprehensive assessments of instruction data. 2) Due to the inherent difference between inter- and intra-sample metrics like diversity and quality, existing methods often heuristically consider each in separate stages with manual thresholds to select data, hindering the dynamic balance of different aspects. 3) Lack of feedback to adaptively refine the selection focus. To address the challenges above, we first establish data selection criteria based on three distinct aspects: diversity, difficulty, and dependability, corresponding to inter-sample, sample-model, and intra-sample perspectives of data value, respectively. Building upon these criteria, we propose D 3 , an effective data selection method consisting of two key steps: scoring and selection. In the scoring step, specifically, to cope with the commonly observed context-oriented generation diversity (CoGD) in text generation, we introduce the uncertainty-based prediction difficulty (UPD), a novel score that quantifies the prediction difficulty of each sample by mitigating the interference caused by CoGD. For diversity and dependability, we define the diversity function based on sample embeddings and use a teacher model to assess the sufficiency and reliability of both the sample’s instruction and its annotated response. In the selection step, we formulate the D 3 weighted coreset object, which jointly optimizes the three aspects of the data value to solve for the most valuable data subset. Additionally, the scoring and selection steps can iterate through multiple rounds. This allows for adaptive refinement of the selection focus by incorporating sample-level feedback from the model during instruction tuning. We conducted extensive experiments on three instruction tuning datasets to validate the D 3 method. Remarkably, with only 5% to 10% of the entire dataset, our approach achieves competitive or even superior performance compared to full-dataset fine-tuning. This highlights the feasibility of D 3 for sample-efficient instruction tuning, which enhances both efficacy and efficiency, thereby advancing related research. To summarize, our main contributions are as follows: (1) Problem : We establish data selection criteria based on three distinct aspects: diversity, difficulty, and dependability, corresponding to inter-sample, sample-model, and intra-sample perspectives of data value, respectively, which offers systematic guidance for the problem of data selection to enable sample-efficient instruction tuning. (2) Method : We propose the D 3 approach, consisting of two key steps. In scoring, we define the diversity function, introduce the uncertainty-based prediction difficulty and integrate a teacher model for dependability assessment. In selection, we formulate the D 3 weighted coreset object to jointly optimize three criteria and solve for the most valuable data subset. The two steps can iterate multiple rounds to adaptively refine the selection focus. (3) Evaluation : Extensive experiments validate the effectiveness of our proposal, demonstrating its potential for sample-efficient instruction tuning to enhance both efficacy and efficiency, thereby facilitating related research. 2 Related Work 2.1 Instruction Tuning for LLMs Instruction tuning Brown et al. 2020 ; Touvron et al. 2023 has emerged as a widely adopted post-training strategy to equip language models with instruction-following capabilities for various downstream tasks. Early research Khashabi et al. 2020 ; Ye et al. 2021 primarily focused on manually creating task-specific instruction tuning datasets. More recent approaches Honovich et al. 2022 ; Wang et al. 2022 ; Xu et al. 2023 ; Wang et al. 2023 ; Taori et al. 2023 shift toward constructing instruction tuning datasets by leveraging powerful LLMs to generate data. These studies facilitate the automatic data creation via distilling knowledge from the teacher model, and are also related to the growing field of LLM-based data synthesis Wang et al. 2024a ; Long et al. 2024 . These studies have laid the foundation for instruction tuning, demonstrating that large task-specific or cross-task instruction tuning datasets can boost the instruction-following performance of language models. 2.2 Sample-Efficient Tuning However, large instruction tuning datasets are often plagued by concerns over quality and redundancy. Some studies Zhou et al. 2023 have suggested that a small, manually curated set of high-quality data can endow the model with powerful instruction-following capabilities. To improve both the performance and efficiency of instruction tuning, recent research attempts to select valuable subsets for training from large datasets. These efforts generally focus on quality and diversity assessments. Du et al. 2023 employs a reward model

Read the complete text