Optimal Interactive Learning on the Job via Facility Location Planning
Shivam Vats 2 1 Michelle Zhao 3 1 Patrick Callaghan 3 Mingxi Jia 2 Maxim Likhachev 3 Oliver Kroemer 3 George Konidaris 2 · 2025
Read Optimal Interactive Learning on the Job via Facility Location Planning by Shivam Vats 2 1 Michelle Zhao 3 1 Patrick Callaghan 3 Mingxi Jia 2 Maxim Likhachev 3 Oliver Kroemer 3 George Konidaris 2 (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-00490.txt.
Introduction
Shivam Vats and co-authors present a planner that helps robots learn on the job with less human effort in 2025. Partner robots must pick up new tasks and user tastes over time. Asking the human too often breaks the flow of shared work, above all across many tasks.
Their planner, called COIL, picks among skill, preference, and help queries to cut total human effort. With known tastes, the choice maps to a classic facility location problem, which fast approximation methods can solve well. A belief step extends the plan when tastes are unsure. Tests in simulation and on real arms cut human work while tasks still got done.
Catalog record
| CREATOR | Shivam Vats 2 1 Michelle Zhao 3 1 Patrick Callaghan 3 Mingxi Jia 2 Maxim Likhachev 3 Oliver Kroemer 3 George Konidaris 2 |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 68 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-00490
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Optimal Interactive Learning on the Job via Facility Location Planning Shivam Vats 2 1 Michelle Zhao 3 1 Patrick Callaghan 3 Mingxi Jia 2 Maxim Likhachev 3 Oliver Kroemer 3 George Konidaris 2 Abstract Collaborative robots must continually adapt to novel tasks and user preferences without overburdening the user. While prior interactive robot learning methods aim to reduce human effort, they are typically limited to single-task scenarios and are not well-suited for sustained, multi-task collaboration. We propose COIL (Cost-Optimal Interactive Learning)—a multi-task interaction planner that minimizes human effort across a sequence of tasks by strategically selecting among three query types (skill, preference, and help). When user preferences are known, we formulate COIL as an uncapacitated facility location (UFL) problem, which enables bounded-suboptimal planning in polynomial time using off-the-shelf approximation algorithms. We extend our formulation to handle uncertainty in user preferences by incorporating one-step belief space planning, which uses these approximation algorithms as subroutines to maintain polynomial-time performance. Simulated and physical experiments on manipulation tasks show that our framework significantly reduces the amount of work allocated to the human while maintaining successful task completion. 0 0 footnotetext: Correspondence to [email protected] . I Introduction Collaborative robots hold the promise of reducing human workload, increasing productivity, and improving quality of life by assisting with tedious and labor-intensive tasks. For human-robot collaboration to be truly effective, robots must safely complete assigned tasks according to individual user preferences. Although robots can be equipped with prior knowledge about the user and a library of common skills, they will inevitably face novel situations in practical long-term deployments. How can they learn and adapt on the job when faced with tasks that are beyond their capabilities or when the user preference is unclear? We explore these questions in the context of multi-task domains, such as factories and households, that require collaboration on a variety of tasks during deployment. The central challenge in this setting is to enable robots to learn interactively from humans while ensuring that all tasks are completed with minimum human effort. Prior interactive learning approaches leverage active querying strategies to proactively identify queries that result in maximum uncertainty reduction [ 6 ] , volume removal [ 29 ] or information gain [ 4 ] . Such active learning strategies have been shown to reduce the number of queries. Furthermore, the use of multiple query types, such as demonstrations and preference queries, has been shown to alleviate the teaching burden by accounting for the cost associated with each query type [ 10 ] . However, these approaches focus on single tasks, and therefore do not guarantee minimization of human effort across multiple tasks in extended collaboration. The latter challenges the robot to reason proactively about the expected utility of its queries so that it does not overburden the user by seeking to learn all the tasks. Fig. 1: A human-robot team working together to pick objects off a conveyor and pack them in bins. The robot queries the human online to learn motor skills and user preference about how each object should be grasped and where it should be placed. We propose a planning algorithm COIL that optimizes how the robot allocates tasks and uses these different types of queries to minimize human effort during the course of its deployment. We propose a novel multi-task interaction planner COIL (Cost-Optimal Interactive Learning) that uses three types of queries— skill , preference , and human help —to complete a given sequence of tasks, simultaneously satisfying hidden user preferences and minimizing user burden. Our key theoretical contribution is a novel formulation of multi-task interactive robot learning as an instance of the uncapacitated facility location (UFL) problem. Different from prior works, our formulation is cost-optimal and jointly minimizes human burden during learning and deployment. Our formulation has significant theoretical and practical implications. Theoretically , it shows that interactive robot learning in our setting can be approximated, i.e, a bounded sub-optimal solution can be computed in polynomial time. Practically , it enables the use strong off-the-shelf UFL solvers for planning. We believe that our UFL formulation opens exciting avenues for future research by leveraging rich theory and algorithms developed by the operations research community, without the need to develop solutions from scratch. Furthermore, we extend our UFL formulation to incorporate multiple query types by proposing a simple but effective one-step belief space planning algorithm that can request user preferences when uncertain. Notably, even solving this one-step belief space planning problem is NP-hard. Our UFL formulation enables us to leverage fast approximation algorithms as sub-routines to plan efficiently. We extensively evaluate and analyze the behavior of our algorithm in simulated and physical manipulation domains. We find that COIL minimizes user burden significantly better than existing approaches, can adapt to failures in teaching, and efficiently scales to long-duration collaboration. COIL results in interactions with 12\%-20\% and 23\% lower cost than the best performing baseline in simulated manipulation experiments and physical conveyor experiments respectively. Additional details and videos of our experiments are available at https://sites.google.com/view/optimal-interactive-learning . II Related Work Learning through multiple interaction modalities is an active area of robotics research [ 14 , 23 ] . Bıyık et al. 2022 first passively collect demonstrations to initialize a belief about the human’s reward function and then actively probe the user with preference queries to zero-in on their true reward. Fitzgerald et al. also aim to learn a human’s reward function, but they do so by actively selecting the query from among Corrections, Demonstrations, Preference Queries, and Binary Rewards expected to be most informative to the learner. In a similar vein, Mehta and Losey 2024 use multiple modalities of physical interaction queries to first learn a reward model online using a neural network representation and then apply constrained optimization to identify an optimal trajectory. Like our work, these works aim to learn from multiple interaction types. However, the aforementioned methods apply to singular tasks, and while they actively query the human for feedback that will be useful to the robot (and can do so while considering the cost of querying), they do not plan for the future utility of learning those tasks in the way our method does. Human preference is another wide-spread topic of interest in robotics [ 5 , 24 , 25 , 26 , 32 ] . Jeon et al. 2020 present a unifying formalism for preference-based learning algorithms through multiple types of interaction with the understanding that a “preference” is implicit in the feedback provided by the human and the skill being taught. Moreover, these preferences must be learned through multiple interaction instances. In contrast, our work encodes preferences as discrete state or action parameters by which a skill must abide and are learned through just one interaction instance. Hadfield-Menell et al. introduce cooperative inverse reinforcement learning to learn a human’s reward function (i.e., preference). While their POMDP-based approach could be applied in our paradigm, our use of a HiP-MDP [ 9 ] makes the problem tractable and enables us to design an efficient, effective planner. Work like Bajcsy et al. isolates a user’s preferences by learning features one-at-a-time until converging to the complete reward model, but they do so solely through demonstration queries and without considering future utility. Finally, reinforcement