Towards Balanced Behavior Cloning from Imbalanced Datasets Journal: Autonomous Robots
Sagar Parekh, Heramb Nemlekar, and Dylan P. Losey · 2025
Read Towards Balanced Behavior Cloning from Imbalanced Datasets Journal: Autonomous Robots by Sagar Parekh, Heramb Nemlekar, and Dylan P. Losey (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-06319.txt.
Introduction
Sagar Parekh, Heramb Nemlekar, and Dylan Losey tackle lopsided demonstration data in robot learning in a 2025 paper. Human demos over represent easy subtasks, and standard cloning weights every sample alike, so rare but key moves get swamped. The team proves that skew in data yields skew in the learned policy.
Reweighting the data set can restore balance, but each automatic method trades something off. The paper sorts those trade offs so others can pick by need. A new meta gradient reweighting method then targets the main flaws of earlier ones. Robot experiments show the rebalanced policies performing better overall than equal weight training on the raw demos.
Catalog record
| CREATOR | Sagar Parekh, Heramb Nemlekar, and Dylan P. Losey |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 81 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2508-06319
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Towards Balanced Behavior Cloning from Imbalanced Datasets Journal: Autonomous Robots Sagar Parekh, Heramb Nemlekar, and Dylan P. Losey Abstract Robots should be able to learn complex behaviors from human demonstrations. In practice, these human-provided datasets are inevitably imbalanced : i.e., the human demonstrates some subtasks more frequently than others. State-of-the-art methods default to treating each element of the human’s dataset as equally important. So if — for instance — the majority of the human’s data focuses on reaching a goal, and only a few state-action pairs move to avoid an obstacle, the learning algorithm will place greater emphasis on goal reaching. More generally, misalignment between the relative amounts of data and the importance of that data causes fundamental problems for imitation learning approaches. In this paper we analyze and develop learning methods that automatically account for mixed datasets. We formally prove that imbalanced data leads to imbalanced policies when each state-action pair is weighted equally; these policies emulate the most represented behaviors, and not the human’s complex, multi-task demonstrations. We next explore algorithms that rebalance offline datasets (i.e., reweight the importance of different state-action pairs) without human oversight. Reweighting the dataset can enhance the overall policy performance. However, there is no free lunch: each method for autonomously rebalancing brings its own pros and cons. We formulate these advantages and disadvantages, helping other researchers identify when each type of approach is most appropriate. We conclude by introducing a novel meta-gradient rebalancing algorithm that addresses the primary limitations behind existing approaches. Our experiments show that dataset rebalancing leads to better downstream learning, improving the performance of general imitation learning algorithms without requiring additional data collection. See our project website: https://collab.me.vt.edu/data_curation/ . ∎ Figure 1: Robot learning how to open a drawer and move a slider from offline human demonstrations. Standard imitation learning equally weights each state-action pair. This results in a policy that imitates the dominant movement patterns seen in the dataset. When the dataset represents one behavior more commonly, imitation learning will learn that behavior at the cost of the other. However, the underrepresented behaviors may not necessarily be unimportant. For instance, opening a drawer and moving the slider require the robot to execute very different motions in different parts of the state space. When the dataset has disproportionately more demonstrations for opening drawer, by imitating the dominant behavior the robot learns to open the drawer but can struggle with moving the slider. So, how do we learn a balanced policy from imbalanced datasets? 1 Introduction Human teachers have multiple tasks that they want their robots to learn. Humans therefore provide a dataset of examples, showing the different tasks and their desired behavior across these tasks. Within the human’s examples there are some movement patterns that show up frequently, and other patterns that are only demonstrated once or twice. For instance, consider a human teaching a robot arm to organize a desk (see Figure 1 ). This may require the robot to pick up objects, open drawers or cabinets, and then put away those objects. On the one hand, there are certain behaviors — like reaching for the desired item — that are commonly observed across all the human’s demonstrations. On the other hand, there are alternate behaviors that are seen rarely in the demonstrations, but are nevertheless important for the robot to learn. For instance, after grasping the desired object, the three tasks — opening a drawer, moving a slider, picking up an object — all require the robot to move in different directions. More generally, when learning complex or multi-part tasks, robot learners are often provided with imbalanced datasets. Some subtasks are overrepresented in these datasets, while other subtasks are underrepresented. For instance, driving datasets predominantly contain everyday on-road conditions with very few rare yet safety-critical scenarios, such as occluded and jaywalking pedestrians samadi2024good . As we will show — when using standard imitation learning algorithms — robots trained on imbalanced datasets learn to focus on the commonly represented movement patterns, often ignoring the infrequent behaviors. Returning to our desk example: if the robot sees a disproportionately large number of demos for opening the drawer, but very few for moving the slider, the robot may overfit to the drawer task while failing to move the slider (see Figure 1 ). This is a practical issue even when the robot is learning a single task. Take the case of picking up objects: if majority of the human’s demonstrations are for picking up objects that are placed on the left side of the table, the common behavior would be for the robot arm to move to towards the left. A robot trained on this dataset could grasp the common behavior (e.g., left objects), but struggle when it encounters objects on the right. Other instances of data imbalance include learning tasks with constraints (e.g., reaching a goal while avoiding obstacle regions), or when learning from multiple human teachers with different skill levels (e.g., some users provide optimal demonstrations while others are noisy). This issue has been known to practitioners and is particularly relevant to recent efforts in collecting large-scale robot datasets containing multiple tasks o2024open ; khazatsky2024droid . However, there is little theoretical understanding of how data imbalance affects the robot’s policy and how it can be effectively mitigated — practitioners often balance their data manually based on intuition to ensure equal learning of all behaviors team2024octo . In this paper, we theoretically investigate the fundamental problem of imitation learning from imbalanced datasets. We begin by showing that the problem arises from a key assumption in standard imitation learning algorithms that each data point contributes equally to the learning process. Specifically, we theorize that: Equally weighing each state-action pair in an imbalanced dataset leads to imbalanced learning. We then explore different strategies for weighing the data samples to learn a more balanced policy. Our analysis indicates that the optimal balance depends on two factors: the relative difficulty of learning the behaviors in the dataset and how accurately we want to learn each behavior. Based on these findings, we highlight the limitations of current methods for automatically balancing robot datasets and introduce a new meta-gradient approach to address them. Overall, we make the following contributions: Formalizing Behaviors. We formalize the dataset of demonstrations as a collection of sub-policies. Each sub-policy represents distinct behaviors required to complete the overall task. For a robot to perform effectively, it must learn each sub-policy regardless of any imbalance in the dataset. Theoretical Analysis of Imbalanced Learning. We theoretically demonstrate that training on an imbalanced dataset leads to policies that are biased towards more frequently occurring behaviors. We validate these findings through experiments, empirically showing that imbalance can impair the robot’s ability to generalize across all behaviors in the dataset. Important Considerations for Balancing Datasets. We provide a theoretical examination of existing approaches that balance datasets and demonstrate experiments highlighting their limitations. Further, we provide insights into the important considerations for balancing datasets. Meta-Gradient Method for Balancing. To overcome the limitations of prior methods, we introduce a meta-gradient approach for balancing datasets. This method iteratively calculates the best possible performance achievable for each behaviors and leverages that to balance the dataset, enabling the policy to learn rare behaviors