Zero-Shot Transferable Solution Method for Parametric Optimal Control Problems
Xinjian Li, Kelvin Kan, Deepanshu Verma, Krishna Kumar, Stanley Osher, Ján Drgoňa · 2025
Read Zero-Shot Transferable Solution Method for Parametric Optimal Control Problems by Xinjian Li, Kelvin Kan, Deepanshu Verma, Krishna Kumar, Stanley Osher, Ján Drgoňa (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-18404.txt.
Introduction
Control problems ask for the best action plan, but the goal often changes from task to task. Xingjian Li and five co-authors present a method that adapts to new goals with no fresh solving. Their 2025 paper learns a set of reusable building blocks for control plans. Once learned, those blocks can be mixed to fit a new task on the fly.
Training happens offline: the blocks are learned by copying expert plans. At run time, only light mixing weights need to be found, from data or from the task description. Tests spanned varied dynamics, sizes, and cost shapes. The method stayed near optimal while adding very little cost per new task. The result is feedback plans that work across a wide range of goals. The authors aim them at real-time use where goals shift often.
Catalog record
| CREATOR | Xinjian Li, Kelvin Kan, Deepanshu Verma, Krishna Kumar, Stanley Osher, Ján Drgoňa |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 36 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2509-18404
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Zero-Shot Transferable Solution Method for Parametric Optimal Control Problems Xinjian Li, Kelvin Kan, Deepanshu Verma, Krishna Kumar, Stanley Osher, Ján Drgoňa Abstract This paper presents a transferable solution method for optimal control problems with varying objectives using function encoder (FE) policies. Traditional optimization-based approaches must be re-solved whenever objectives change, resulting in prohibitive computational costs for applications requiring frequent evaluation and adaptation. The proposed method learns a reusable set of neural basis functions that spans the control policy space, enabling efficient zero-shot adaptation to new tasks through either projection from data or direct mapping from problem specifications. The key idea is an offline–online decomposition: basis functions are learned once during offline imitation learning, while online adaptation requires only lightweight coefficient estimation. Numerical experiments across diverse dynamics, dimensions, and cost structures show our method delivers near-optimal performance with minimal overhead when generalizing across tasks, enabling semi-global feedback policies suitable for real-time deployment. I INTRODUCTION Optimal control problems arise ubiquitously across engineering disciplines [ 5 , 11 , 40 , 34 ] . Although the fundamental mathematical framework remains consistent, practical applications often require solving parametric problems where objectives vary according to task specifications, such as target locations in trajectory planning, terrain characteristics in mobile robotics, or process requirements in manufacturing. Classical local solution methods [ 39 ] are relatively fast but must be solved anew for each instance, whereas global solution methods based on the Hamilton–Jacobi–Bellman equations [ 16 ] are intractable in high dimensions. Machine learning-based approaches aim to bridge this gap and have achieved considerable success [ 42 , 31 ] ; however, they are typically tied to a fixed objective and lack transferability across tasks. This paper addresses the challenge of efficiently adapting control policies to new objectives without solving each problem instance from scratch. The key insight is to approximate the function space of control policies using a function encoder (FE) [ 23 ] , which learns a reusable set of neural network–parameterized functions. Policies for new tasks are then expressed as linear combinations of these basis functions, with task-specific coefficients inferred in a zero-shot manner either from limited trajectory data (LS) or directly from the problem specification (operator). As such, the computation cost and data requirement for online adaptation are greatly reduced, enabling efficient transfer without sacrificing accuracy. The main contributions of the work are: • An imitation learning-based framework for parametric optimal control problems that allows for zero-shot generalization to unseen problem instances without model retraining. • A semi-global feedback formulation that works for arbitrary inputs and is particularly well-suited when repeated evaluation of the model is required. • Validation through extensive numerical experiments, demonstrating the robustness and near-optimal accuracy on high-dimensional and nonlinear examples. Fig. 1 : Function encoder policies: the online–offline decoupling enables efficient and accurate policy adaptation to different optimal control objectives. II Background This section details the necessary background, covering optimal control and the function encoder framework. Parametric Optimal Control Problem Given an initial state {\bf x}(0)={\bf x}_{0} , we consider a class of optimal control problems where the system dynamics have a fixed form but the objective functional varies with each new task specification. The system evolves according to \dot{{\bf x}}(t)=f({\bf x}(t),{\bf u}(t),t),\quad{\bf x}(0)={\bf x}_{0}, (1) where {\bf x}(t)\in\mathbb{R}^{n} is the state, {\bf u}(t)\in\mathcal{U}\subseteq\mathbb{R}^{m} is the control input, and f:\mathbb{R}^{n}\times\mathbb{R}^{m}\times[0,T]\to\mathbb{R}^{n} is the dynamics function. For each task, defined by a conditional variable {\boldsymbol{\eta}} , the objective functional in parametric form is \min_{{\bf u}\in\mathcal{U}}\mathcal{J}({\bf u};{\boldsymbol{\eta}})=\int_{0}^{T}L({\bf x}(t),{\bf u}(t),t;{\boldsymbol{\eta}})\,dt+G({\bf x}(T);{\boldsymbol{\eta}}), (2) where L:\mathbb{R}^{n}\times\mathbb{R}^{m}\times[0,T]\to\mathbb{R} is the running cost, G:\mathbb{R}^{n}\to\mathbb{R} is the terminal cost, and \mathcal{U} is the space of admissible control functions, which we assume to be sufficiently regular [ 16 , Sec. I.3, I.8-9 ] . For simplicity and without loss of generality, we use {\boldsymbol{\eta}} to denote the conditional variable that captures task-specific variations in \mathcal{J} , such as target locations, terrain types, or changes in control penalties. In practice, {\boldsymbol{\eta}} can be complex, high-dimensional or implicit. Each choice of {\boldsymbol{\eta}} defines a distinct optimal control problem. Given the open-loop definition in ( 2 ), lets consider a closed-loop feedback form [ 30 ] given as {\bf u}({\bf x}(t),t;{\boldsymbol{\eta}}):\mathbb{R}^{n}\times[0,T]\to\mathcal{U}\subseteq\mathbb{R}^{m}, (3) for each specific task. Having the control {\bf u} to depend on both the current state {\bf x}(t) and time t allows for control evaluation for any state-time pair, making it more flexible and particularly useful in problems where the initial state {\bf x}_{0} can vary. Classical results show that under suitable regularity assumptions, existence of optimal controls in feedback form can be guaranteed; see, e.g., [ 16 , Sec. I.5] , or [ 6 ] . Function Encoder Function encoder (FE) [ 23 ] provides a principled framework for representing and transferring tasks in Hilbert spaces by learning a finite set of neural network basis functions. Given a Hilbert space \mathcal{H} , the FE learns basis functions \{\phi_{1},\phi_{2},\dots,\phi_{p}\} , parameterized by neural networks, such that any function f\in\mathcal{H} can be approximated as f({\bf x})\approx\sum_{j=1}^{p}c_{j}\phi_{j}({\bf x};{\boldsymbol{\theta}}_{j}), (4) for some {\bf c}:=[c_{1},c_{2},\dots,c_{p}]^{\top}\in\mathbb{R}^{p} . Here we use {\boldsymbol{\theta}}:=\{{\boldsymbol{\theta}}_{1},{\boldsymbol{\theta}}_{2},\dots,{\boldsymbol{\theta}}_{p}\} to denote the trainable parameters of the basis functions. This linear representation enables inductive transfer of tasks: new functions are represented by inferring the coefficients {\bf c} directly from task-specific measurements without retraining the basis functions. Online computation of coefficients for a given new problem f is obtained by solving a least-squares (LS) problem {\bf c}=\argmin_{{\bf c}\in\mathbb{R}^{p}}\|f-\sum_{j=1}^{p}c_{j}\phi_{j}\|_{\mathcal{H}}^{2}. (5) This admits a closed-form solution given by {\bf c}=\begin{bmatrix}\langle\phi_{1},\phi_{1}\rangle_{\mathcal{H}}&\cdots&\langle\phi_{1},\phi_{p}\rangle_{\mathcal{H}}\\ \vdots&\ddots&\vdots\\ \langle\phi_{p},\phi_{1}\rangle_{\mathcal{H}}&\cdots&\langle\phi_{p},\phi_{p}\rangle_{\mathcal{H}}\end{bmatrix}^{-1}\begin{bmatrix}\langle f,\phi_{1}\rangle_{\mathcal{H}}\\ \vdots\\ \langle f,\phi_{p}\rangle_{\mathcal{H}}\end{bmatrix}, (6) where \langle\cdot,\cdot\rangle_{\mathcal{H}} denotes the inner product associated with the Hilbert space \mathcal{H} , a common choice is the L^{2} inner product. Both the matrix, known as the Gram matrix, and the right-hand side vector of ( 6 ) can be computed directly from task-related measurements or data, using Monte Carlo integration. In practice, we also incorporate Tikhonov regularization to ( 5 ) to improve numerical stability; see [ 17 ] . Importantly, FE are supported by a strong theoretical guarantee. Theorem 1 (Universal Function Space Approximation [ 23 ] ) . Let K\subset\mathbb{R}^{n} be compact and let \mathcal{H}=\{f:K\to\mathbb{R}^{n}\mid\|f\|_{\mathcal{H}}<\infty\} be a separable Hilbert space. For any continuous f\in\mathcal{H} and any \epsilon>0 , there exist neural network basis functions \{\phi_{j}\}_{j=1}^{P} and coefficients \{c_{j}\}_{j=1}^{P} such that \Big\|f-\sum_{j=1}^{P}c_{j}\,\phi_{j}\Big\|_{\mathcal{H}}<\epsilon\,\|f\|_{\mathcal{H}}. Here Theorem 1 establishes that, with a sufficient number of basis functions, FEs can approximate any function in \mathcal{H} with arbitrary precision, making them a principled and general-purpose tool for our transfer learning task at hand. III Transferable Solution Method for Parametric Optimal Control Problems Many practical optimal control applications require repeatedly solving problems where system dynamics remain fixed but objectives vary. For example, in trajectory planning, the target destination may change from one instance to another, giving rise to a class of problems that must be solved repeatedly. While existing approaches (see Section IV ) can achieve high accuracy on a fixed problem setting, changes in the objective typically require recomputation of solutions from scratch, leading to substantial computational overhead. The central challenge hindering model adaptability is to efficiently approximate the family of control policies \{{\bf u}^{*}(\cdot,\cdot;{\boldsymbol{\eta}})\}_{{\boldsymbol{\eta}}} without re-solving each problem instance. To address this, we propose a methodology that directly targets task variability. Our approach leverages the function encoder framework to learn a transferable policy representation,