Mission-Aligned Learning-Informed Control of Autonomous Systems: Formulation and Foundations
Vyacheslav Kungurtsev, Gustav Šír · 2025
Read Mission-Aligned Learning-Informed Control of Autonomous Systems: Formulation and Foundations by Vyacheslav Kungurtsev, Gustav Šír (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-04356.txt.
Introduction
Vyacheslav Kungurtsev, Gustav Šír, and co-authors set out a control frame for autonomous systems, in a paper from twenty twenty-five. It unites learning, planning, and control. Spending on autonomous body agents, from robots to drones, is rising fast. Their test scenario is a stylized version of robotic care. Such care usually trains one reinforcement learning policy for motion. It trains another for conceptual tasks.
The frame swaps that setup for a two-level optimization scheme. Control theory governs the lower level of body motion. Classical planning handles the higher level of tasks and sub-tasks, joined with learning. The authors present background and the general statement of the frame. They detail how each part is joined. Trust here means body safety plus interpretability for users and those who regulate. The synergy, they argue, should yield leaner and more reliable autonomous results.
Catalog record
| CREATOR | Vyacheslav Kungurtsev, Gustav Šír |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 119 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2507-04356
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Mission-Aligned Learning-Informed Control of Autonomous Systems: Formulation and Foundations VK and GS are with Czech Technical University in Prague. AA and SG are with the Norwegian University of Science and Technology. HT and HH are with Imperial College London. Funding support is acknowledged to the Prosocial AI STEM initiative through the Institute for Advanced Conscious Studies in Los Angeles, CA, USA. Vyacheslav Kungurtsev, Gustav Šír Abstract Research, innovation and practical capital investment have been increasing rapidly toward the realization of autonomous physical agents. This includes industrial and service robots, unmanned aerial vehicles, embedded control devices, and a number of other realizations of cybernetic/mechatronic implementations of intelligent autonomous devices. In this paper, we consider a stylized version of robotic care, which would normally involve a two-level Reinforcement Learning procedure that trains a policy for both lower level physical movement decisions as well as higher level conceptual tasks and their sub-components. In order to deliver greater safety and reliability in the system, we present the general formulation of this as a two-level optimization scheme which incorporates control at the lower level, and classical planning at the higher level, integrated with a capacity for learning. This synergistic integration of multiple methodologies—control, classical planning, and RL—presents an opportunity for greater insight for algorithm development, leading to more efficient and reliable performance. Here, the notion of reliability pertains to physical safety and interpretability into an otherwise black box operation of autonomous agents, concerning users and regulators. This work presents the necessary background and general formulation of the optimization framework, detailing each component and its integration with the others. 1 Introduction Consider a setting in which a robot is a permanent aide to a movement and health-impaired human. This robot is supposed to provide for the individual’s needs, while generally attempting to benefit their health and well-being. It should also ensure that the individual is safe, and that any unexpected behavior should trigger immediately notifying a human specialist with the most critical information. There are various possible relevant objects in the environment, and the robot either moves next to the human to assist movement or retrieves the object for the human. There are a set of regular expected activities, eating, grooming, exercising, scribing dictation, etc. that the robot performs in accordance with: the desires of the human, the human’s well-being as according to the responsible health authorities, the physical safety of the human, and any regulations. The robot can communicate and should suggest any activity on the schedule to the human, who can refuse or stop the activity at any time. Warnings can be issued, e.g., “if we move away now your living room will have this pile and may look messy,” but ultimately the human decides its behavior. In service of this task, the robot is able to lean to roll forward and back as well as turn in full rotation, and rotate its arm along three axes. Assume, for simplicity, that objects can be picked up without modeling grasping, as if everything relevant was magnetized. Objects in the environment define collisions to be avoided, which requires including the speed and position in the state space. The robot has a propulsion system, which is its forward control, and a braking mechanism. The manufacturing of this propulsion system involves some new features which are not system-identified to full certainty, and there is a small but noticeable nonlinear gyroscopic effect on its motion. This effect will have to be learned in the course of its operation. This work will present a formalism that defines a bilevel optimization problem incorporating Reinforcement Learning (RL)-assisted Model Predictive Control in the bottom layer, and RL-assisted classical planning at the higher level. In describing the formalism we will return to this example, and in ongoing works analyzing the theoretical properties and testing a simulation of this stylized example. Human-aligned and safe autonomous robotics is a technology of significant interest to many researchers, companies, and governments, with population aging creating many potential labor shortages for assistance care and health for the elderly, the potential for automated manufacturing, and other reasons. As it is also a representative example of the conceptual components of such an operation, it will be a running example throughout this line of work. In this work we simplify some of these concerns and focus on developing the foundations of the mathematical formulation for the optimization framework. We present a formal schema to define the autonomous decision procedures of an agent using multiple relevant disciplines associated with mathematical engineering. AI and Autonomous Systems Designing and managing systems towards optimality or looser satisfaction criteria over time is a generic cross-disciplinary problem that has attracted attention from numerous communities. In the case wherein the model of the states and actions can be well represented by a finite set while the system dynamics and the utility of different circumstances is unknown, Reinforcement Learning (RL) has risen to prominence as the preeminent methodology. RL, like Classification (e.g., image recognition), and Generative Learning (e.g., Large Language Models), has been a central component in the accelerating, groundbreaking developments of AI that have characterized the technological landscape for the past decade. Neural networks leverage the raw computational power of highly parallel GPU hardware permitting training of deep models with millions to billions of trainable parameters. This forms highly accurate statistical models of objects, processes and phenomena when there are vast quantities of data present or available. In recent years, a variety of approaches for opening the deep and opaque models developed by AI training have appeared aiming to improve their interpretability and reliability. In the case of image recognition and LLMs, these concerns are largely a matter of ensuring an agreeable user experience. However, RL involves making decisions. In the context of games of pure entertainment, this relative lack of criticality of risks persists. However, when RL agents make decisions in contexts with real-world consequences, a notable degree of justified concern appears. An opaque black-box model simply cannot provide formal guarantees of safety or performance reliability. Rather, it is simply a model that efficiently provides a most likely answer, in some formal probabilistic sense. Low probability but catastrophic events could happen. Contributions The general premise of this work is as follows: it is currently difficult if not impossible to ensure that RL can systematically avoid or minimize the risk of damaging living beings or critical agents and infrastructure via action or instability. Probabilistic learning techniques, importance sampling in unsafe regions, human in the loop, and other strategies that remain within the RL paradigm garcia2015comprehensive may present clever mechanisms to dramatically lower the probability of said risks, but they are fundamentally insufficient. Comprehensive guarantees of safe and stable operation must arise from a paradigm that is separate from probabilistic inductive learning. In particular, crisp hard science-based models of system dynamics and foreseeable risks (e.g. of collision) are the standard, and a comprehensively reliable approach to endow RL systems with true guarantees of safe and stable operation. Said guarantees would significantly expand the set of domains and circumstances to which RL can be applied, and at the same time provide a transparent and interpretable means by which regulators can certify the safety of an RL system/agent. This will be done