Heavy-Ball Dynamics with Hessian-Driven Damping for Non-Convex Optimization under the Łojasiewicz Condition
Vassilis Apidopoulos c,∗, Vasiliki Mavrogeorgou ⋄,∗, Theodoros G. Tsironis ♯,∗ · 2025
Read Heavy-Ball Dynamics with Hessian-Driven Damping for Non-Convex Optimization under the Łojasiewicz Condition by Vassilis Apidopoulos c,∗, Vasiliki Mavrogeorgou ⋄,∗, Theodoros G. Tsironis ♯,∗ (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-11705.txt.
Introduction
Vassilis Apidopoulos, Vasiliki Mavrogeorgou, and Theodoros G Tsironis study a physics-inspired method for finding low points of rough functions. The heavy-ball method rolls a virtual ball downhill with momentum, and extra damping based on the curve's bend can settle it faster. Their 2025 paper treats smooth but non-convex problems that meet a growth rule named the Łojasiewicz condition. They ask how the friction settings shape the worst-case speed of the method.
Their rates are tight and beat earlier ones, even for strongly curved problems. The math also suggests a fresh way to tune the damping terms. Extra results cover noisy inputs and rough gradients. The team shows the method also slips past strict saddle points from almost any start.
Catalog record
| CREATOR | Vassilis Apidopoulos c,∗, Vasiliki Mavrogeorgou ⋄,∗, Theodoros G. Tsironis ♯,∗ |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 76 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2506-11705
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Heavy-Ball Dynamics with Hessian-Driven Damping for Non-Convex Optimization under the Łojasiewicz Condition Vassilis Apidopoulos c,∗, Vasiliki Mavrogeorgou ⋄,∗, Theodoros G. Tsironis ♯,∗ Abstract . In this paper, we examine the convergence properties of heavy-ball dynamics with Hessian-driven damping in smooth non-convex optimization problems satisfying a Łojasiewicz condition. In this general setting, we provide a series of tight, worst-case optimal convergence rate guarantees as a function of the dynamics’ friction coefficients and the Łojasiewicz exponent of the problem’s objective function. Importantly, the linear rates that we obtain improve on previous available rates and they suggest a different tuning of the dynamics’ damping terms, even in the strongly convex regime. We complement our analysis with a range of stability estimates in the presence of perturbation errors and inexact gradient input, as well as an avoidance result showing that the dynamics under study avoid strict saddle points from almost every initial condition. \subcaptionsetup labelfont=bf,justification=centering 2020 Mathematics Subject Classification Primary 90C26, 34D05, 46N10; secondary 65K05, 65B99. 1. Introduction Consider the minimization problem (Opt) \displaystyle\operatorname*{minimize}_{x\in\mathcal{H}} \displaystyle F(x) where {\color[rgb]{0,0,0}F}\colon\mathcal{H}\to\mathbb{R} is a C^{2} -smooth (not necessarily convex) function on some Hilbert space \mathcal{H} . Our aim in this paper is to study the asymptotic minimization and convergence properties of the dynamical inertial Newton-like ( DIN ) system (DIN) \ddot{x}(t)+\alpha\dot{x}(t)+\beta\nabla^{2}F(x(t))\dot{x}(t)+\nabla F(x(t))=0 where \alpha,\beta\geq 0 are the system’s friction coefficients – inertial and Hessian respectively. System ( DIN ) was introduced and studied in [ 2 ] and its origins can be traced back to the seminal work of Polyak [ 45 ] (see also [ 4 , 1 , 10 ] ) for the case \beta=0 – in which case ( DIN ) is typically referred to as the HBF ( HBF ) dynamics: (HBF) \ddot{x}(t)+\alpha\dot{x}(t)+\nabla F(x(t))=0 The key feature of system ( DIN ) is the incorporation of the Hessian-driven friction term \nabla^{2}F(x(t))\dot{x}(t) , to the classical heavy ball with friction dynamics ( HBF ) ( [ 45 , 4 , 1 , 10 ] ). The consideration of such Hessian damping term makes the generated trajectory more adaptive to the local curvature of the objective function F and mitigates the oscillations that may appear in the behavior of the heavy ball with friction trajectory in ( HBF ), which may have undesirable effects from an optimization objective. One of the major interests of studying inertial systems like ( DIN ) is their accelerated convergence properties (e.g. in terms of speed of convergence of F(x(t)) to a critical value \bar{F} ), with respect to first-order in time systems like the standard gradient flow : (GF) \dot{x}(t)+\nabla F(x(t))=0. by properly tuning the friction parameters \alpha and \beta . In general, the quantitative properties of systems like ( DIN ) and more precisely the question concerning their rates of convergence (and thus their acceleration properties) strongly depends on i) the precise optimization framework (i.e. the assumptions made on the geometry/regularity F ) and ii) the adequate tuning of the friction parameters \alpha and \beta , in order to achieve the best possible convergence results. In fact, the question of acceleration for the system ( DIN ) beyond the (strongly) convex realm, under milder assumptions such as the Łojasiewicz condition or Kurdyka-Łojasiewicz condition [ 44 , 34 , 33 ] still remains unclear and is the main subject of this work. Contributions In this work, we aim to address the aforementioned issue for the system ( DIN ). In particular, we study the convergence properties of ( DIN ) in a non-convex setting (when F is not necessarily convex), under the Łojasiewicz condition [ 34 , 45 ] (see Assumption 2 in Section 2 ). More precisely, when F satisfies the \mu -Łojasiewicz condition of order 2 , we provide explicit exponential rates of convergence for F(x(t)) to a critical value \bar{F} and for the decay of \nabla F(x(t)) to zero, as a function of the friction parameters \alpha and \beta . As a by-product, we provide an explicit tuning for the friction parameters, leading to an optimal worst-case rate of order \mathcal{O}(e^{-2\sqrt{\mu}t}) , which is improved with respect to previous related works [ 2 , 8 , 26 , 37 , 49 ] for the system ( DIN ). Our theoretical improvements are also witnessed in some simple numerical illustrations. We additionally provide stability estimates for the trajectory of ( DIN ) in the presence of an external perturbation error, that depend both on the local landscape of the objective function F , around its critical points (i.e. the Łojasiewicz exponent of F ) and the tuning of the parameters \alpha , \beta . Such stability results present some interest on their own when it comes to the consideration of stochastic versions of ( DIN ) where the computation of \nabla F (or \nabla^{2}F ) is not (or hardly) accessible. Furthermore, we complement the analysis by establishing an almost-surely avoidance of strict saddle points result, for functions satisfying the Łojasiewicz condition, further generalizing upon existing works [ 24 ] (see also [ 37 ] ) for functions with isolated critical points. Finally, we extend the above-mentioned results to the first-order in time coupled system ( g-DIN ), which is related to ( DIN ) (see below in paragraph 1.1.2 ), paving the way to the consideration of more generalized settings, where the objective function F is not necessarily smooth or even differentiable. 1.1. Related work 1.1.1. Heavy-ball with friction System ( HBF ) was firstly introduced and studied in the seminal work of Polyak [ 45 ] , showing its advantages with respect to the gradient flow dynamics ( GF ). In particular, for twice differentiable and \mu -strongly convex functions the ( HBF ) system with \alpha=2\sqrt{\mu} enjoys the worst-case optimal linear rate of convergence of order \mathcal{O}(e^{-2\sqrt{\mu}t}) , in terms of objective function values F(x(t))-\min F , while the corresponding rate for ( GF ) is \mathcal{O}(e^{-2\mu t}) . For the majority of practical functions, it is most likely that the strong convexity parameter is very small (i.e., \mu\ll 1 ), making ( HBF ) much faster than ( GF ). In this viewpoint the order \mathcal{O}(e^{-c\sqrt{\mu}t}) , for some constant c\leq 2 is usually identified as an accelerated linear rate. Beyond the (strongly) convex setting, several studies have been devoted to the convergence properties of ( HBF ), including i) almost sure convergence to local minima for (non convex) Morse functions [ 10 , 30 ] and ii) convergence rates for the heavy ball trajectories under various relaxations of strong convexity, such as quasi-strong convexity [ 46 , 13 , 16 , 14 ] and Łojasiewicz conditioning (see Assumption {\color[rgb]{0,0,0}\mathcal{L}}(2) ) [ 31 , 17 , 43 , 5 ] . However, it is worth mentioning that, to the best of the authors knowledge, the worst-case optimal rate \mathcal{O}(e^{-2\sqrt{\mu}t}) , obtained by ( HBF ) for strongly convex functions, has not been established for any of the aforementioned relaxations (see also the related discussion in Remark 4 in Section 3 ). 1.1.2. Dynamic Inertial Newton-like system While the heavy ball system ( HBF ) enjoys some nice convergence properties, its trajectory often presents oscillatory behavior, which may cause numerical instabilities and affect the overall efficiency