GCImOpt : Learning efficient goal-conditioned policies by imitating optimal trajectories

Jon Goikoetxea, Jesús Palacián · 2026

Read GCImOpt : Learning efficient goal-conditioned policies by imitating optimal trajectories by Jon Goikoetxea, Jesús Palacián (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-22724.txt.

Introduction

Goikoetxea and Palacián teach goal-based control rules by copying paths made with optimization, not by gathering costly human examples. Their GCImOpt plan makes thousands of best paths in minutes on a laptop and grows the set by treating middle states as goals. The final rules stay small, win often, and run fast enough for on-board control chips.

Catalog record

CREATORJon Goikoetxea, Jesús Palacián
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE41168
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.22724

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

GCImOpt : Learning efficient goal-conditioned policies by imitating optimal trajectories Jon Goikoetxea, Jesús Palacián Abstract Imitation learning is a well-established approach for machine-learning-based control. However, its applicability depends on having access to demonstrations, which are often expensive to collect and/or suboptimal for solving the task. In this work, we present GCImOpt , an approach to learn efficient goal-conditioned policies by training on datasets generated by trajectory optimization. Our approach for dataset generation is computationally efficient, can generate thousands of optimal trajectories in minutes on a laptop computer, and produces high-quality demonstrations. Further, by means of a data augmentation scheme that treats intermediate states as goals, we are able to increase the training dataset size by an order of magnitude. Using our generated datasets, we train goal-conditioned neural network policies that can control the system towards arbitrary goals. To demonstrate the generality of our approach, we generate datasets and then train policies for various control tasks, namely cart-pole stabilization, planar and three-dimensional quadcopter stabilization, and point reaching using a 6-DoF robot arm. We show that our trained policies can achieve high success rates and near-optimal control profiles, all while being small (less than 80 000 80\,000 neural network parameters) and fast enough (up to more than 6000 × 6000\times faster than a trajectory optimization solver) that they could be deployed onboard resource-constrained controllers. We provide videos, code, datasets and pre-trained policies under a free software license; see our project website https://jongoiko.github.io/gcimopt/ . \SetKwComment Comment/* */ keywords Machine learning, imitation learning, goal conditioning, optimal control, trajectory optimization Figure 1: Schematic summary of GCImOpt , a simple method to train and evaluate near-optimal goal-conditioned policies. A policy \bm{\pi}_{\theta} , parameterized by a neural network, is trained by supervised learning on a large dataset of trajectories generated by trajectory optimization. The policy is then evaluated in simulation to measure its success rate and efficiency. 1 Introduction Optimally controlling a dynamical system enables the accomplishment of a task while minimizing a given cost measure. However, for many dynamical systems, finding an optimal closed-loop controller or policy is very difficult or impossible. For this reason, trajectory optimization ( 10 ; 4 ) is often used in practice, for example as part of model predictive control (MPC) ( 27 ) : the controller computes open-loop optimal controls in a loop at high frequency, executing at each step the first segment of the optimal control. Even though MPC allows the design of near-optimal closed-loop controllers, solving optimization problems at high frequency makes it computationally expensive. In addition to obtaining controllers that are efficient and have low computational cost, we are interested in having them be general : a general policy would be able to accomplish multiple control tasks on the system without having to modify the policy itself. This can be obtained by passing a representation of a goal to the controller so that its output is conditioned both on the current system state and the desired goal. Generalization is a central problem in machine learning (ML), since we aim to train models that perform well on the entire data distribution, even when faced with previously unseen data. Thus, a data-driven method can be effective to obtain near-optimal goal-conditioned policies that have low computational cost at inference: using trajectory optimization to generate a large dataset of optimal controls, we can train a small neural network to approximate the optimal controller and generalize to arbitrary goals. 1.1 Related work Goal conditioning Goal-conditioned policies have been extensively studied in the context of reinforcement learning ( 17 ; 33 ; 22 ) . A fundamental advantage of goal conditioning is that trajectories can be relabeled to have their goals be different to the originally intended goals ( 2 ) , achieving greater sample efficiency than direct RL on goal-conditioned tasks. In spite of this, GCRL approaches often require careful reward shaping and online environment interaction, introducing the challenge of exploration; these difficulties can be avoided by resorting to imitation learning. In goal-conditioned imitation learning (GCIL), we require datasets where the expert solves multiple tasks, which may then be used to speed up training RL agents ( 11 ) , use the expert’s intentions as goals ( 8 ) or imitate multimodal experts with expressive policy classes ( 28 ) . Additionally, GCIL may be scaled to use massive and heterogeneous datasets, enabling the training of general models that work on various robot morphologies ( 34 ) . GCIL also relaxes the need for expert behavior in the dataset, meaning that suboptimal, uncurated data can still be leveraged for policy learning ( 23 ; 14 ) . Imitation learning on optimal trajectories Trajectories obtained by trajectory optimization can be used as expert data for imitation learning. Earlier work makes a careful consideration of the interplay between policy learning and trajectory optimization, e.g. penalizing the optimized trajectories’ deviation from the policy’s outputs ( 24 ; 21 ; 40 ; 18 ) or leveraging the structure of system dynamics to ensure the policy’s stability ( 9 ) . In recent years, there has been a renewed interest in learning from optimal trajectories computed offline, with special emphasis on aerospace applications ( 31 ; 32 ; 30 ; 35 ) . A notable characteristic of these works is the simplicity of the training setup, since policies are directly trained using behavioral cloning rather than using a more complex algorithm such as DAgger ( 29 ) . 36 attribute this advantage to low input dimensionality and large dataset sizes, which minimize the impact of compounding errors in the policies’ predictions. Policies trained using this method have been deployed in real quadcopters ( 13 ) , where they have been shown to perform comparably to differential flatness-based controllers. Although behavior cloning of policies on optimal trajectories has been extensively studied, little work has been done on extending the generality of said policies by means of goal conditioning. 1.2 Contributions The contributions of this work are the following: • We present GCImOpt , a simple method to train and evaluate near-optimal goal-conditioned policies on datasets generated by trajectory optimization (Fig. 1 ). Unlike GCRL methods, our method does not require reward shaping or online environment interaction, since training is done on optimal trajectories computed offline. We expand on previous work by considering the effects of goal conditioning, thus training policies that can generalize to arbitrary goals while remaining near-optimal. We use a direct method for trajectory optimization, thus avoiding the need to analytically construct optimality conditions for each considered control problem; in addition, we make dataset generation fast by using the Fatrop solver ( 37 ) , which is purpose-built for optimal control applications. • To demonstrate the generality of GCImOpt , we evaluate it empirically on four different simulated dynamical systems. We thus also evaluate the impact of the system’s dimensionality and nonlinearity on the trained policies’ performance. We show that GCImOpt policies can achieve high success rates and near-optimal control profiles. Moreover, we show that GCImOpt policies can achieve speedups of 97\times to 6278\times versus the fast Fatrop solver, suggesting that they could surpass the control frequency of MPC controllers. • We release an implementation of the method, the code of the experiments, the generated datasets

Read the complete text