State-dependent temperature control in Langevin diffusions using numerical exploratory Hamiltonian-Jacobi-Bellman equations
Taorui Wang, Xun Li, Gu Wang, Zhongqiang Zhang · 2026
Read State-dependent temperature control in Langevin diffusions using numerical exploratory Hamiltonian-Jacobi-Bellman equations by Taorui Wang, Xun Li, Gu Wang, Zhongqiang Zhang (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-17934.txt.
Introduction
Wang, Li, Wang, and Zhang tune noise levels inside Langevin dynamics by solving control equations. The right noise size makes or breaks hard optimization runs. Exploratory Hamilton-Jacobi-Bellman equations can set that noise state by state. High dimensions stalled past work on bounds and Hessian math, though. Their physics-informed network bakes the equation shape into training for stability. Tests stayed robust far past one-dimensional cases, they report.
Catalog record
| CREATOR | Taorui Wang, Xun Li, Gu Wang, Zhongqiang Zhang |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 56396 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.17934
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
State-dependent temperature control in Langevin diffusions using numerical exploratory Hamiltonian-Jacobi-Bellman equations Taorui Wang, Xun Li, Gu Wang, Zhongqiang Zhang Abstract Choosing how much noise to add in Langevin dynamics is essential for making these algorithms effective in challenging optimization problems. One promising approach is to determine this noise by solving Hamilton–Jacobi–Bellman (HJB) equations and their exploratory variants. Though these ideas have been demonstrated to work well in one dimension, extension to high-dimensional minimization has been limited by two unresolved numerical challenges: setting reliable control bounds and stably computing the second‑order information (Hessians) required by the equations. These issues and the broader impact of HJB parameters have not been systematically examined. This work provides the first such investigation. We introduce principled control bounds and develop a physics‑informed neural network framework that embeds the structure of exploratory HJB equations directly into training, stabilizing computation, and enabling accurate estimation of state‑dependent noise in high‑dimensional problems. Numerical experiments demonstrate that the resulting method remains robust and effective well beyond low‑dimensional test cases. keywords stochastic optimization, exploratory control, entropy‑regularized HJB equations, gradient descent methods, scientific machine learning, high‑dimensional PDEs 1 Introduction In gradient descent methods for optimization, one common strategy to escape local minima and saddle points is the addition of noise, leading to Langevin dynamics. Designing such noise terms, however, is nontrivial and often problem-dependent. From a theoretical perspective, noise can be controlled through diffusion terms, with the optimal control derived from the Hamilton–Jacobi–Bellman (HJB) equation (see the workflow below). Yet, this classical approach often yields extreme (bang–bang) noise levels, which can be rigid and numerically brittle [ 27 , 2 ] . To address this limitation, Gao et al. [ 11 ] proposed a randomized control with entropy regularization. Their method, termed the exploratory HJB equation, demonstrated clear benefits in one-dimensional problems, such as escaping local minima of a double-well function. Tang et al. [ 29 ] further established that the exploratory HJB equation converges to the classical HJB equation as the regularization vanishes. \min_{x\in\mathbb{R}^{d}}f(\bm{x}) Langevin dynamics Hamilton–Jacobi–Bellman (HJB) equation regularization, parameters, solvers design control (temperature) Despite these advances, the extent to which exploratory HJB formulations provide reliable state-dependent temperature schedules in higher dimensions remains insufficiently understood, particularly in landscapes with numerous saddle points and competing local minima. Furthermore, scalable solvers for both classical and exploratory HJB equations are still under active development. Classical discretization-based approaches have been investigated in low dimensions, e.g., in [ 9 ] , while deep learning methods have been explored in low and high dimensions, e.g., in [ 20 , 19 , 21 , 7 ] . The underlying computational difficulty stems from strong nonlinearity combined with the need to resolve solutions on high-dimensional domains. In this work, we develop practical numerical methods for solving eHJB equations to obtain efficient state-dependent Langevin dynamics for minimization problems. The work differs from the literature in that our goal is to obtain accurate state-dependent noise coefficients that depend on the Laplacian of the value functions—which solve the eHJB equation—rather than on the value functions themselves, e.g. in [ 19 ] . Three practical issues arise in the development of numerical methods: (i) A reliable evaluation of the Laplacian (trace of the Hessian), and (ii) stable computation of the log-partition term -\log\!\int_{\mathcal{U}}\exp(-\frac{u}{\lambda}(\cdot))\,du , approximating the minimum over \mathcal{U} and (iii) the range of control for the noise magnitude or temperature. First, the evaluation of the temperature requires the Hessian of the solution to the HJB (eHJB) equations, which is continuous but not differentiable since the eHJB solution in [ 11 ] is locally C^{2,\alpha} , \alpha\in(0,1) . Hence, additional regularization is essential. We adopt physics-informed neural networks (PINNs, [ 26 ] ) to solve the eHJB equations, relying on their implicit regularization under stochastic gradient–based training. We show numerically that PINNs trained with the LION optimizer [ 4 ] yield solutions of eHJB equations that provide effective state-dependent temperatures for nonconvex optimization. The above approach addresses the issue of (i). It is observed that accurate eHJB solutions are not required for effective optimization. Based on Theorem 4.1 , the convergence of temperature is slow in the exploration parameter \lambda and the training error. Fortunately, we observe in Section 5 that the performance of the Langevin dynamics with eHJB primarily depends on the relative shape of the Laplacian rather than its precise values: nearly zero noise around global minimizers, and larger noise near local extrema and saddle points. PINNs provide smoothed approximations that capture the correct qualitative structure even when the underlying solution is not fully resolved. Second, we develop an efficient stabilized implementation of the log-partition that avoids numerical integration while maintaining accuracy. The log-partition term has a closed-form expression for an interval \mathcal{U} , but direct evaluation can be unstable due to the exponential and logarithm functions. When their arguments are close to zero, we use Taylor’s approximation to address (ii) (see Section 1 ). Moreover, we use a truncation argument if the noise magnitude or temperature is very close to zero (see Section 4.4 ). Third, we adopt a simple but effective scaling rule to determine the control set \mathcal{U}=[u_{\min},u_{\max}] , instead of manual selection [ 11 ] . While the noise coefficient or temperature may remain positive, it may induce oscillations near global minimizers. To address this issue, we truncate the noise coefficients when they are below a dyadic fractions of \sqrt{2u_{\max}} without harming the efficiency or the accuracy. See more details in Section 4.3 . In summary, we apply PINNs for exploratory HJB equations and use the Laplacian of the solutions to place the state-dependent temperature and noise coefficient. The novelty and key contributions of the work are listed below: 1. From theory and illustration in 1D to genuinely nonconvex minimization in higher dimensions. We extend the computation using eHJB-based state-dependent temperature control from one dimension [ 11 ] to d=2 – 6 , where saddle points and local minimum dominate the optimization landscape. The purpose of applying PINNs here is not to solve the eHJB equation pointwise, but to learn the structure of its Laplacian being small in the vicinity of global minimizers and large near saddle points and local minima. This perspective fundamentally differs from most existing HJB solvers, including the recent eHJB solver of [ 19 ] , which targets accurate pointwise solutions. In addition, our method solves the eHJB equation offline, requiring only a single run of PINNs, whereas [ 19 ] solves it on the fly and therefore invokes PINNs repeatedly. 2. A stable algorithm for elevating the nonlinear operator in eHJB. We design an efficient, numerically stable implementation of the nonlinear log-partition operator in the eHJB equation, eliminating the need for numerical quadrature. See Section 1 for details. This algorithm allows stable computation in PINNs for eHJB directly and thus avoids the policy iterations and multiple runs of PINNs. 3. Control-range design and robust Langevin dynamics. We provide in Section 4.3 an empirical rule for selecting the control bounds that set the diffusion/noise coefficient (rather than hand-tuning as in [ 11 ] ), and we stabilize the resulting dynamics with mirror reflection at the boundary of the computational domain and truncation of the noise coefficient to suppress oscillations near global minimizers while