Two - Stage Learning of Stabilizing Neural Controllers via Zubov Sampling and Iterative Domain Expansion

Haoyu Li, Xiangru Zhong, Bin Hu, Huan Zhang · 2025

Read Two - Stage Learning of Stabilizing Neural Controllers via Zubov Sampling and Iterative Domain Expansion by Haoyu Li, Xiangru Zhong, Bin Hu, Huan Zhang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-01356.txt.

Introduction

Li, Zhong, Hu, and Zhang teach neural network controllers to stabilize systems with guarantees, in a 2025 paper. Learned controllers perform well but rarely come with proof they will not destabilize the system, or any map of the states from which they can recover. Their two-stage method trains a controller together with a Lyapunov function, the mathematical certificate of stability.

A sampling strategy based on Zubov's characterization sketches the true stability boundary, and the training region expands step by step, cutting wasted caution. Verification uses an extended neural network checker rather than a slow logic solver. The resulting safe regions are vastly larger than baselines', with far faster checks.

Catalog record

CREATORHaoyu Li, Xiangru Zhong, Bin Hu, Huan Zhang
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE65 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2506-01356

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Two - Stage Learning of Stabilizing Neural Controllers via Zubov Sampling and Iterative Domain Expansion Haoyu Li, Xiangru Zhong, Bin Hu, Huan Zhang Abstract Learning-based neural network (NN) control policies have shown impressive empirical performance. However, obtaining stability guarantees and estimations of the region of attraction of these learned neural controllers is challenging due to the lack of stable and scalable training and verification algorithms. Although previous works in this area have achieved great success, much conservatism remains in their framework. In this work, we propose a novel two-stage training framework to jointly synthesize the controller and Lyapunov function for continuous-time systems. By leveraging a Zubov - inspired region of attraction characterization to directly estimate stability boundaries, we propose a novel training data sampling strategy and a domain updating mechanism that significantly reduces the conservatism in training. Moreover, unlike existing works on continuous-time systems that rely on an SMT solver to formally verify the Lyapunov condition, we extend state-of-the-art neural network verifier α , β \alpha,\!\beta -CROWN with the capability of performing automatic bound propagation through the Jacobian of dynamical systems and a novel verification scheme that avoids expensive bisection. To demonstrate the effectiveness of our approach, we conduct numerical experiments by synthesizing and verifying controllers on several challenging nonlinear systems across multiple dimensions. We show that our training can yield region of attractions with volume 5 − 1.5 ⋅ 10 5 5-1.5\cdot 10^{5} times larger compared to the baselines, and our verification on continuous systems can be up to 40 − 10000 40-10000 times faster compared to the traditional SMT solver dReal. Our code is available at https://github.com/Verified-Intelligence/Two-Stage_Neural_Controller_Training . 1 Introduction Learning-based neural network control policies have demonstrated great potential in complex systems due to their high expressiveness ( zhang2016learning ; wang2020reinforcement ; kaufmann2023champion ; zhao2024applications ) . However, the applications of these neural controllers in safety-critical real-world physical systems are of concern since they typically lack stability guarantees. A promising method for quantifying safety is estimating the region of attraction (ROA), which refers to the set of initial states from which a system is guaranteed to converge to a desired equilibrium point. Computation of the ROA is often formulated as the search for an appropriate Lyapunov function satisfying algebraic conditions over its sub-level sets ( lyapunov1992general ; valmorbida2017region ; ratschan2010providing ) . Due to the highly complex and nonlinear nature of neural networks, searching for a Lyapunov function is not easy. Many existing works that are able to make stability guarantees about a system typically have some restrictions on the problem structure, for example, requiring linear or polynomial parameterizations of the system, where sum-of-squares technique ( tedrake2010lqr ; yang2023approximate ; papachristodoulou2002construction ; dai2023convex ) can be applied. To address this limitation and enable certification for general nonlinear systems, many recent works explore data-driven methods to synthesize a neural certificate, such as Lyapunov functions ( chang2019neural ; dai2021lyapunov ; zhou2022neural ; wu2023neural ; yang2024lyapunov ; shi2024certified ; liu2025physics ) , Barrier Functions ( ames2016control ; robey2020learning ; dai2023convex ; dawson2022safe ; liu2023safe ; dawson2022learning ) , and Contraction Metrics ( tsukamoto2020neural ; tsukamoto2020stochastic ; tsukamoto2021theoretical ; dawson2023safe ; sun2021learning ) , to certify the system’s stability. In this work, we aim to synthesize neural controllers of continuous-time systems together with a Lyapunov certificate. Unlike typical machine learning tasks, this problem requires that the Lyapunov and boundary conditions hold over the entire continuous domain of interest , rather than statistically achieving high accuracy on a discrete test set. However, since at each iteration only a small number of points can be selected, the problem forces us to adopt specialized strategies both to pick the training domain and to sample training data from the domain. We argue that these two are the key factors for the conservativeness issue presented in many previous works. For the training domain selection, many previous works simply select a fixed domain of interest and enforce the Lyapunov condition over it. While some previous works proposed a new formulation to overcome the conservativeness by introducing an extra term to verify forward invariance and allow the ROA to touch the training domain non-trivially ( yang2024lyapunov ; shi2024certified ) , we argue that this does not fully solve the conservativeness issue and leaves the training domain selection a hard hyperparameter to tune. In addition to the training domain, training data selection also plays an important role. Besides simple random sampling that typically does not suffice for certification, previous works dealing with similar problems ( chang2019neural ; dai2021lyapunov ; zhou2022neural ; yang2024lyapunov ) typically achieve co-learning with a counterexample-guided (CEGIS) approach, which generates counterexamples of the Lyapunov condition either using a verifier ( chang2019neural ; dai2021lyapunov ) such as an SMT or MILP solver or a gradient-based approach ( wu2023neural ; yang2024lyapunov ) to be the training data. While these approaches have led to great success, such approaches can typically only refine upon already worked controller/Lyapunov function pair and thus rely on carefully tuned LQR/RL based initializations ( chang2019neural ; wu2023neural ; yang2024lyapunov ) . This limitation makes these works both conservative in their estimation of the ROA and also very difficult to apply to new systems without much domain-specific knowledge. In our work, we propose a two-stage training framework with carefully designed curriculum domain expansion and training sample selection that can lead to much more non-conservative ROA estimation inspired by Zubov theorem ( zubov1961methods ; liu2025physics ; kang2023data ; meng2024zubov ) ,which provides a characterization of the true ROA with some PDE. During the first stage, we propose a Zubov-inspired training data sampling strategy by combining points from inside and outside of the current ROA estimation to better guide the learning. Moreover, we eliminate the conservativeness introduced by the training domain by dynamically expanding it using trajectory information. This stage can provide us an almost working controller and Lyapunov function that we can refine upon. The second stage then involves counterexample-guided learning (CEGIS) to eliminate all the counterexamples inside the ROA estimation and thus obtain a verifiable Lyapunov certificate. As a by-product, our training also fully utilizes the physics-informed Zubov loss and can eliminate the conservatism introduced by the inner approximation nature of the normal Lyapunov function. As we shall demonstrate in our numerical experiments, our approach can robustly produce a controller with a significantly larger ROA compared to previous state-of-the-art approaches in all the benchmarks with various control input limits. As we deal with continuous-time systems, verification over Jacobian vector products also presents its challenge. Previous works addressing the Lyapunov stability verification for continuous-time systems typically rely on SMT or SDP based solvers ( chang2019neural ; abate2020formal ; liu2025physics ; chen2021learning ; yin2021stability ; fazlyab2020safety ; chen2021learning ) such as dReal ( gao2013dreal ) . Inspired by the success in ( yang2024lyapunov ; shi2024certified ) , we also adapt the state-of-the-art neural network verification tool \alpha,\!\beta -CROWN ( zhang2018efficient ; salman2019convex ; xu2020automatic ; xu2021fast ; wang2021beta ; zhang2022general ) for formally verifying Lyapunov stability. The key challenge is to handle the Jacobian-related operators in dynamical systems, which have not been discussed or implemented in existing neural network verifiers. To ensure verification tightness, we design new linear relaxations for these Jacobian operators in continuous time system learning such

Read the complete text