Active Constraint Learning in High Dimensions from Demonstrations
Zheng Qiu, Chih-Yuan Chiu, Glen Chou · 2025
Read Active Constraint Learning in High Dimensions from Demonstrations by Zheng Qiu, Chih-Yuan Chiu, Glen Chou (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2512-22757.txt.
Introduction
Zheng Qiu, Chih-Yuan Chiu, and Glen Chou tackle a robotics problem: a robot watching demonstrations must infer the hidden constraint that shaped them, such as a region the demonstrator was avoiding. In high dimensions, random demonstrations waste effort because most reveal little about the constraint. Their active constraint learning algorithm instead asks for the demonstrations it expects to be most informative.
The method fits a Gaussian process model of the unknown constraint to the demonstrations seen so far. It then uses that model's uncertainty to pick start and goal states likely to teach it something new. Fresh demonstrations from those states are added and the loop repeats. In simulations and on real hardware with nonlinear dynamics and nonlinear constraints, the approach infers constraints more accurately than a random-sampling baseline while using sparse, well-chosen examples.
Catalog record
| CREATOR | Zheng Qiu, Chih-Yuan Chiu, Glen Chou |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 55 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2512-22757
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Active Constraint Learning in High Dimensions from Demonstrations Zheng Qiu, Chih-Yuan Chiu, Glen Chou Abstract We present an iterative active constraint learning (ACL) algorithm, within the learning from demonstrations (LfD) paradigm, which intelligently solicits informative demonstration trajectories for inferring an unknown constraint in the demonstrator’s environment. Our approach iteratively trains a Gaussian process (GP) on the available demonstration dataset to represent the unknown constraints, uses the resulting GP posterior to query start/goal states, and generates informative demonstrations which are added to the dataset. Across simulation and hardware experiments using high-dimensional nonlinear dynamics and unknown nonlinear constraints, our method outperforms a baseline, random-sampling based method at accurately performing constraint inference from an iteratively generated set of sparse but informative demonstrations. keywords Learning from demonstration, Constraint inference, Gaussian Processes, Robot safety. Figure 1 : Given demonstrations (orange) generated via a 7-DOF robot arm, our GP-ACL algorithm recovers a nonlinear obstacle set with fewer instances of false positive (FP, in green) and false negative (FN, in red) errors, compared to a random-sampling baseline method. 1 Introduction Recently, learning from demonstrations (LfD) via inverse optimal control (IOC) has emerged as a versatile framework for empowering robots to infer the constraints underlying an expert demonstrator’s actions ( Chou et al. 2022 ; Menner et al. 2021 ; Chou et al. 2020b ; Armesto et al. 2017 ; Papadimitriou and Li 2023 ) , with existing methods proving effective at inferring unknown constraints from a given set of approximately locally-optimal demonstrations. However, current approaches implicitly assume that the demonstrations supplied for training are generated in a manner agnostic to the downstream constraint learning task, and thus the supplied data may not always be informative for reducing constraint uncertainty. As such, existing constraint inference approaches may be data-inefficient, requiring a large number of demonstrations to accurately learn unknown constraints. To overcome this limitation, we present an iterative active constraint learning algorithm built upon the Gaussian process (GP)-based constraint learning framework in Chou et al. 2022 , which uses the Karush-Kuhn-Tucker (KKT) optimality conditions of the provided demonstrations to train nonparametric GP-based constraint representations. At each iteration, our algorithm trains a Gaussian Process (GP) on a given dataset of locally-optimal demonstrations to represent the unknown constraint as a GP posterior. Although active learning methods have been widely explored in the context of cost inference, to our knowledge, our work provides the first framework for active constraint inference over continuous state spaces. Our contributions are: 1. Leveraging the GP-based constraint learning framework in Chou et al. 2022 , we sample multiple GP posterior estimates of an unknown constraint given a demonstration dataset \mathcal{D} . The generation of multiple GP posterior samples provides a more complete description of the constraint information encoded in \mathcal{D} , as well as the remaining constraint uncertainty that cannot be resolved using \mathcal{D} . 2. We present a GP-based active constraint learning algorithm, GP-ACL (Alg. 11 ), which iteratively queries start/goal constraint states to induce the generation of informative demonstrations that reduce constraint uncertainty. 3. We evaluate our GP-ACL algorithm by inferring high-dimensional, nonlinear constraints from the generated demonstrations, both via 4D unicycle and 12D quadcopter dynamics in simulation, and via a 7-DOF robot arm hardware platform. Related Works Existing LfD-based methods have enabled constraint learning via both IOC ( Chou et al. 2020a ; Chou et al. 2022 ; Armesto et al. 2017 ; Papadimitriou et al. 2022 ; Menner et al. 2021 ) and inverse reinforcement learning frameworks ( McPherson et al. 2021 ; Stocking et al. 2022 ; Papadimitriou and Brown 2024 ; Singh et al. 2018 ). Specifically, Chou et al. 2020a ; Chou et al. 2022 ; Papadimitriou and Li 2023 use KKT optimality conditions corresponding to the provided demonstrations to formulate inverse optimization problems, from which constraint information can then be extracted. In particular, our work builds upon and is most similar to Chou et al. 2022 , which likewise embeds the KKT optimality conditions for the provided demonstrations into a Gaussian process (GP) framework to represent unknown constraints. However, whereas Chou et al. 2022 only computes the mean and covariance functions of a trained GP posterior for constraint inference, we additionally sample GP posterior samples as surrogate functions for the unknown constraint map, to facilitate the efficient generation of informative demonstrations. Moreover, while existing constraint learning methods can enable downstream planning that is robust to epistemic uncertainty ( Chou et al. 2022 ; Chou et al. 2021 ) , they do not actively seek to reduce uncertainty through soliciting informative demonstrations. In contrast, our GP-ACL algorithm explicitly actively generates demonstrations guided by the downstream constraint learning objective, a capability not considered in prior work. Prior work has also considered the problem of active intent inference, with the aim of extracting the unknown reward or cost of an expert demonstrator from a provided set of trajectory demonstrations. In particular, methods have been developed to achieve active information gathering ( Sadigh et al. 2016 ; Sadigh et al. 2018 ; Li et al. 2025b ) , intent demonstration ( Li et al. 2024 ) , and uncertainty reduction ( Mesbah 2018 ; Hu and Fisac 2023 ) for human-robot interaction tasks. Meanwhile, Akrour et al. 2012 ; Fang et al. 2017 ; Lee et al. 2021 devise active reward learning methods for reinforcement learning (RL). Our work likewise considers the purposeful generation of demonstrations that are maximally informative with respect to a downstream inference task. However, unlike the works listed above, our GP-ACL algorithm leverages demonstration data to recover unknown constraints , rather than unknown intent, rewards, or costs. Finally, our methods are related to recently developed RL-based ( Papadimitriou et al. 2022 ) and GP-based ( Li et al. 2025a ) active constraint learning methods. Unlike these works, however, our method is capable of inferring unknown high-dimensional constraints in continuous and infinite state and constraint spaces and does not require both constraint-satisfying and constraint-violating behavior to guide learning, which is important for safety-critical robotics applications. 2 Preliminaries and Problem Formulation 2.1 Demonstration Generation and KKT Optimality Conditions By a demonstration , we refer to a state-control trajectory \xi:=(x_{1}(\xi),\cdots,x_{T}(\xi),u_{1}(\xi),\cdots,u_{T}(\xi))\in\mathbb{R}^{(n+n_{i})T} of length T\in\mathbb{N} , where x_{t}(\xi)\in\mathbb{R}^{n} and u_{t}(\xi)\in\mathbb{R}^{n_{i}} respectively denote the state system and control vector identified with the demonstration \xi at each time t\in[T]:=\{1,\cdots,T\} . As in Sec. III-A in Chou et al. 2022 , we assume that each demonstration is a locally-optimal solution to the constrained optimization problem described below (Prob. 1 ). First, let c:\mathbb{R}^{(n+n_{i})T}\rightarrow\mathbb{R} , \textbf{g}_{k}:\mathbb{R}^{(n+n_{i})T}\rightarrow\mathbb{R}^{N_{k}^{\textrm{ineq}}} , and \textbf{h}_{k}:\mathbb{R}^{(n+n_{i})T}\rightarrow\mathbb{R}^{N_{k}^{\textrm{eq}}} respectively encode a (possibly non-convex) cost function, as well as a set of known inequality and equality constraints. In our work, we focus on smoothness-based costs of the form c(\xi):=\sum_{t=1}^{T-1}\|x_{t+1}-x_{t}\|_{2}^{2} ; similar costs are often used in the constraint learning literature to encode trajectory length minimization ( Chou et al. 2022 ; Papadimitriou and Li 2023 ). Moreover, the known equality constraint \textbf{h}_{k}(\cdot)=0 encodes a set of deterministic, nonlinear dynamics x_{t+1}=f_{t}(x_{t},u_{t}),\forall\ t\in[T] , as well as constraints on the initial and final system states, which we describe in more detail in Sec. 2.4 . Next, to formulate constraints unknown to the learner, let \phi_{\text{sep}}:\mathbb{R}^{n}\rightarrow\mathbb{R}^{n_{c}} denote a map from each system state x to a constraint state \phi_{\text{sep}}(x)\in\mathbb{R}^{n_{c}} ,