In-situ Value-aligned Human-Robot Interactions with Physical Constraints

Hongtao Li, Ziyuan Jiao, Xiaofeng Liu, Hangxin Liu, Zilong Zheng · 2025

Read In-situ Value-aligned Human-Robot Interactions with Physical Constraints by Hongtao Li, Ziyuan Jiao, Xiaofeng Liu, Hangxin Liu, Zilong Zheng (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-07606.txt.

Introduction

Hongtao Li and co-authors teach home robots to work in line with human values while respecting physical limits, in this 2025 paper. Language-model robots can now do many household tasks once thought out of reach. Doing the task, though, is not enough: people have preferences about how it is done, and those tastes should carry to new situations. At the same time, arms and objects obey hard physical rules that no preference can bend.

The team built a benchmark of everyday home activities judged partly by preference. Their learning method takes feedback as direct instructions and as corrections during the act. The robot must weigh both what the person wants and what physics allows. The framework aims at helpers that feel less like tools and more like considerate partners.

Catalog record

CREATORHongtao Li, Ziyuan Jiao, Xiaofeng Liu, Hangxin Liu, Zilong Zheng
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE32 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2508-07606

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

In-situ Value-aligned Human-Robot Interactions with Physical Constraints Hongtao Li, Ziyuan Jiao, Xiaofeng Liu, Hangxin Liu, Zilong Zheng Abstract Equipped with llm , human-centered robots are now capable of performing a wide range of tasks that were previously deemed challenging or unattainable. However, merely completing tasks is insufficient for cognitive robots, who should learn and apply human preferences to future scenarios. In this work, we propose a framework that combines human preferences with physical constraints, requiring robots to complete tasks while considering both. Firstly, we developed a benchmark of everyday household activities, which are often evaluated based on specific preferences. We then introduced iclhf ( iclhf ), where human feedback comes from direct instructions and adjustments made intentionally or unintentionally in daily life. Extensive sets of experiments, testing the iclhf to generate task plans and balance physical constraints with preferences, have demonstrated the efficiency of our approach. I Introduction Equipping robots, especially service robots, with the ability to consider personalized human preferences is a challenging task. On the one hand, this is due to the subjectivity and diversity of human preferences, and on the other hand, the physical constraints of the objective world limit the realization of preferences. Imagine a scenario where robots tidy up a table, as shown in Fig. 1 , where humans expect the robot to tidy up according to their preferences. Therefore, it is inappropriate to consider preferences without regard to physical constraints, or physical constraints only, but only to take both into account, i.e., behaving in accordance with human preferences while adhering to physical constraints. Fig. 1: An example task of robots tidying up a table with human preferences. (a) The messy table that needs to be cleared. (b) Considering human preferences without regard to physical constraints would result in unrealistic behavior. (c) Considering physical constraints alone would fail to meet human expectations. (d) Example of human preferences. (e) Only by balancing both, that is, behaving following human preferences while adhering to physical constraints, can the task be satisfactorily completed. One of the challenges is to learn human preferences, with existing methods mainly including learning by pairwise comparison [ 1 , 2 ] and learning by llm [ 3 , 4 , 5 ] . The former typically simplifies complex preferences into a ranking function, which assigns a partial order to multiple outputs of the model. Such methods are easy to implement and have numerous applications in recommendation systems [ 6 , 7 ] , human-robot interaction [ 8 ] , natural language processing [ 9 , 10 ] , and more. However, they require collecting a large amount of annotated data from humans [ 11 , 12 , 13 ] , so what is learned reflects the shared preferences rather than individualized preferences [ 14 ] . Additionally, this form of comparison itself also sacrifices the diversity of human preferences. The latter arises from the powerful text processing and common-sense reasoning capabilities exhibited by llm trained on massive datasets [ 15 , 16 ] . For example, TidyBot [ 3 ] utilizes the summarization capabilities of llm to infer human preferences for tidying up a room, while DROC [ 4 ] learns human preferences from online human language corrections. Another major challenge is the integration of learned human preferences and physical constraints. Physical constraints manifest in both task planning [ 17 ] and motion planning [ 18 ] of robots. In this paper, we only consider the physical constraints of the task planning, and the human preferences associated with it only involve the outcomes of planning. Although it is a subset of the original problem, it remains a challenging task. Due to the inherent, universal, and omnipresent nature of physical constraints, they are typically predefined in the form of hand-written rules, such as the domain description in pddl ( pddl ) [ 19 , 20 ] , or environment modeling in reinforcement learning [ 21 ] . On the other hand, personalized human preferences are often unpredictable, ambiguous, and diverse. The heterogeneity between these two demands an appropriate way of combining them [ 22 ] . Traditional task planning methods, such as those based on pddl or scene graphs, require converting preferences into a form recognizable by planners, such as preference predicates [ 23 ] or constraints [ 24 ] . However, this conversion often sacrifices the diversity of preferences. Another approach is to use llm to convert text-based human preferences into a reward function in reinforcement learning [ 25 , 5 ] and learn physical constraints through a large number of trial-and-error. However, the intrinsic nature of reinforcement learning requires human preferences, serving as reward signals, to be strongly goal-oriented for specific tasks [ 26 ] , thus making it less suited for learning diverse preferences. To address the aforementioned issues, we first proposed a set of household benchmarks, collecting tasks with strong personal preferences. Then, we introduced the iclhf ( iclhf ) algorithm, which aims to combine llm ’ preference learning capability with the ability to learn from feedback. In this approach, the llm functions like a reinforcement learning policy model, learning human preferences through in-context learning [ 27 , 28 ] . Feedback is provided in textual form, combining physical constraints and human preferences. Finally, the structure like rlhf ( rlhf ) of the algorithm allows for balancing physical feedback and preference feedback to generate suitable solutions. Meanwhile, the employment of in-context learning avoids training an excess of parameters for llm , maximizing its inferential prowess as much as possible, thus enabling in-situ personalized preference learning. To achieve generalization, learned human preferences can be easily combined into a hierarchical structure, with higher-level preferences being more adaptive. Considering the powerful capability of traditional algorithms in handling intricate manipulation tasks, we integrated a customized version of POG [ 29 ] , an algorithm for efficient sequential manipulation planning on scene graphs, to enhance the preliminary plans generated by the llm planner. Consequently, the final task plan incorporates more comprehensive geometric spatial information, thereby ensuring seamless transitions to the motion planner. We conducted numerous experiments to validate the effectiveness of the iclhf algorithm in learning human preferences and combining them with physical constraints. Finally, real robot experiments demonstrate the validation of the approach on robotic hardware. The contribution of our work can be summarized as follows: • We present a benchmark on household activities whose evaluation is based on personalized preferences. • We introduce the iclhf algorithm that learns human preferences in situ and combines them with physical constraints to accomplish the task. • We conduct large-scale experiments to validate the effectiveness of iclhf , and real-world robot experiments to demonstrate its practicality. I-A Related Works I-A 1 LLMs for Robots As llm trained on massive amounts of data exhibit powerful common-sense reasoning capabilities [ 15 , 16 ] , a significant amount of work focuses on utilizing them for robotic task planning [ 25 , 30 , 31 ] . These works can be categorized based on the format of llm ’ output into two types: action primitives based [ 30 , 3 ] methods and code-based [ 25 , 31 , 5 ] methods. Methods based on action primitives guide llm to generate

Read the complete text