Offline Behavioral Data Selection Conference: Proceedings of the 32nd ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.1; August 09–13, 2026; Jeju Island, Republic of Korea Proceedings of the 32nd ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.1 (KDD ’26), August 09–13,
Shiye Lei, Zhihao Cheng, Dacheng Tao · 2025
Read Offline Behavioral Data Selection Conference: Proceedings of the 32nd ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.1; August 09–13, 2026; Jeju Island, Republic of Korea Proceedings of the 32nd ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.1 (KDD ’26), August 09–13, by Shiye Lei, Zhihao Cheng, Dacheng Tao (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2512-18246.txt.
Introduction
Shiye Lei, Zhihao Cheng, and Dacheng Tao trim the training data for robots that learn by copying. Copy based learning, called behavioral cloning, feeds on huge logs of expert moves, and training on all of them is costly. The team finds a sharp saturation: a small slice of the log already gives nearly the full skill.
The slack comes from a mismatch between test loss and real driving skill. Their Stepwise Dual Ranking method picks a compact slice that favors early moves, high value actions, and rarely seen states. On the standard D4RL test suite, policies trained on the picked slice beat those trained the usual way, with far less data to chew through.
Catalog record
| CREATOR | Shiye Lei, Zhihao Cheng, Dacheng Tao |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 41 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2512-18246
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Offline Behavioral Data Selection Conference: Proceedings of the 32nd ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.1; August 09–13, 2026; Jeju Island, Republic of Korea Proceedings of the 32nd ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.1 (KDD ’26), August 09–13, 2026, Jeju Island, Republic of Korea DOI: 10.1145/3770854.3780320 ISBN: 979-8-4007-2258-5/2026/08 CCS: Computing methodologies Learning from demonstrations Shiye Lei, Zhihao Cheng, Dacheng Tao Abstract . Behavioral cloning is a widely adopted approach for offline policy learning from expert demonstrations. However, the large scale of offline behavioral datasets often results in computationally intensive training when used in downstream tasks. In this paper, we uncover the striking data saturation in offline behavioral data: policy performance rapidly saturates when trained on a small fraction of the dataset. We attribute this effect to the weak alignment between policy performance and test loss, revealing substantial room for improvement through data selection. To this end, we propose a simple yet effective method, Stepwise Dual Ranking (SDR), which extracts a compact yet informative subset from large-scale offline behavioral datasets. SDR is build on two key principles: (1) stepwise clip, which prioritizes early-stage data; and (2) dual ranking, which selects samples with both high action-value rank and low state-density rank. Extensive experiments and ablation studies on D4RL benchmarks demonstrate that SDR significantly enhances data selection for offline behavioral data. The code is available at https://github.com/LeavesLei/stepwise_dual_ranking . \acmArticleType Research \acmCodeLink https://github.com/leaveslei/acmart † † cc-license: by 1. Introduction In offline reinforcement learning (RL) ( Levine et al. 2020 ) , where direct interaction with the environment is not feasible, behavioral cloning (BC) offers a simple yet effective supervised approach for learning policies from expert demonstrations ( Pomerleau 1991 ; Bain and Sammut 1995 ) . However, offline behavioral datasets are typically large-scale, limiting the scalability of BC in many downstream tasks. To address this challenge, compressing large datasets into compact yet informative subsets has emerged as a promising direction. These compressed datasets not only facilitate efficient policy fine-tuning across diverse pre-trained models ( Noh et al. [n. d.] ) , but also serve as lightweight replay buffers in continual policy learning, effectively mitigating catastrophic forgetting ( Wolczyk et al. 2022 ) . Numerous dataset compression methods, including coreset selection ( Guo et al. 2022 ; Zhou et al. 2022 ) and dataset distillation ( Wang et al. 2018 ; Zhao et al. 2021 ; Lei and Tao 2024 ) , have been developed to construct smaller yet highly representative training sets. However, these approaches are largely tailored to the supervised learning paradigm, which assumes i.i.d. data and access to clean, perfectly labeled samples. In contrast, offline behavioral datasets violate the i.i.d. assumption and exhibit complex interactions between the data distribution and policy performance, which makes the direct application of existing compression techniques nontrivial. Recent works ( Lei et al. 2024 ; Lei et al. 2025 ) apply dataset distillation to synthesize a compact dataset from large-scale offline behavioral data, although the distillation procedure requires computing Hessian matrices and is therefore computationally demanding. Figure 1. The workflow of Stepwise Dual Ranking. The original dataset is processed through stepwise progressive clip and dual ranking to construct a candidate pool. The final informative subset is then randomly selected from this candidate pool. This paper investigates efficient coreset selection for offline behavioral data, aiming to identify a compact yet informative subset from the original behavioral dataset to enable rapid policy training. By analyzing the relationship between policy performance and training sample size, we uncover a striking phenomenon of data saturation in offline behavioral datasets: policy performance rapidly saturates as more training data is added, and a randomly selected subset containing only 1\%\sim 5\% of the data can train a policy that achieves 90\% of the performance attained using the full dataset. In contrast, supervised image datasets typically require at least 20\% of the data to reach similar performance levels. Furthermore, we demonstrate that this data saturation arises from the weak correlation between optimization loss and policy performance. Specifically, reducing the loss does not necessarily translate into meaningful improvements in policy performance, a sharp contrast to conventional supervised learning. The pronounced data saturation highlights the potential of data selection. Building on a theoretical analysis of per-step allocation and the implicit weighting of behavioral data, we propose a simple yet effective algorithm, Stepwise Dual Ranking (SDR), which constructs a compact and informative subset from offline behavioral datasets. Two main strategies are involved in SDR: (1) stepwise progressive clip: prioritize selecting more data visited in the front steps, while choose less data with large steps; and (2) dual ranking: we rank the examples based on density and action values respectively, and prone to select data with both low density rank and high action value ranks. Furthermore, to mitigate the algorithm sensitivity to estimation error, we employ two-stage sampling to generate the final subset. Concretely, we first use SDR to shape a large candidate pool and then randomly select a small set from the candidate pool. This can enhance data diversity and mitigates sensitivity induced by estimation error on data density and action value; please see Figure 1 for illustration. Extensive experiments on the D4RL benchmark across multiple environments demonstrate that our proposed SDR significantly outperforms the random selection baseline, whereas conventional selection criteria and coreset selection algorithms perform substantially worse in the context of offline behavioral data. Our contributions can be summarized as: • We identify and analyze a significant phenomenon of data saturation in offline behavioral datasets, supported by both empirical and theoretical analysis. • We introduce SDR, a novel and efficient approach for constructing an informative subset from large-scale offline behavioral data. • SDR achieves a substantial performance improvement over both traditional selection baselines and existing coreset selection methods. 2. Related Works Offline Behavioral Cloning Unlike conventional online RL, which requires direct interaction with the environment, offline RL aims to learn policies from a pre-collected dataset, thereby mitigating the costs and risks associated with real-world interactions ( Levine et al. 2020 ) . BC is a widely used offline RL approach that learns policies by mimicking expert demonstrations ( Pomerleau 1991 ; Bain and Sammut 1995 ; Torabi et al. 2018 ) . Since BC directly imitates the expert policy without requiring reward signals, it offers great flexibility and can be applied to a wide range of complex tasks, including autonomous driving ( Bojarski et al. 2016 ; Ly and Akhloufi 2020 ) and robotic control ( Giusti et al. 2015 ; Niekum et al. 2015 ; Daftry et al. 2016 ) . However, offline behavioral datasets for BC are often large-scale, leading to computational inefficiencies in training and limiting its practicality for downstream applications. This challenge highlights the need for efficient data selection techniques to improve the feasibility of BC in real-world scenarios. Coreset Selection Selecting a representative subset from a large dataset is a promising approach to improving data efficiency in model training ( Guo et al. 2022 ) . Coreset selection has been widely applied in various domains, including active learning ( Park et al. 2022 ; Yehuda et al. 2022 ) , continual learning ( Yoon et al. 2022 ; Tiwari et al. 2022 ; Hao et