Accelerated Inertial Gradient Algorithms with Vanishing Tikhonov Regularization
Samir Adly Vinh Thanh Ho Huu Nhan Nguyen Laboratoire XLIM, Université de Limoges, 123 Avenue Albert Thomas, 87060 Limoges CEDEX, France · 2026
Read Accelerated Inertial Gradient Algorithms with Vanishing Tikhonov Regularization by Samir Adly Vinh Thanh Ho Huu Nhan Nguyen Laboratoire XLIM, Université de Limoges, 123 Avenue Albert Thomas, 87060 Limoges CEDEX, France (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-23035.txt.
Introduction
Samir Adly, Vinh Thanh Ho, and Huu Nhan Nguyen study a fast gradient method with a fading regularizer. Their algorithm targets smooth convex minimization, where the objective's gradient changes at a controlled rate. It comes from discretizing a damped inertial system, a model of motion with friction.
A Tikhonov regularizer that vanishes over time steadies the path without biasing the end point. Explicit, step-by-step versions suit real computation better than continuous models. Faster convex solvers sit under logistics, imaging, and machine learning pipelines. Vanishing regularization is the balancing act: strong enough early to guide the iterates, gone at the end so the true solution remains.
Catalog record
| CREATOR | Samir Adly Vinh Thanh Ho Huu Nhan Nguyen Laboratoire XLIM, Université de Limoges, 123 Avenue Albert Thomas, 87060 Limoges CEDEX, France |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 98 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2601-23035
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Accelerated Inertial Gradient Algorithms with Vanishing Tikhonov Regularization Samir Adly Vinh Thanh Ho Huu Nhan Nguyen Laboratoire XLIM, Université de Limoges, 123 Avenue Albert Thomas, 87060 Limoges CEDEX, France Abstract Abstract. In this paper, we study an explicit Tikhonov-regularized inertial gradient algorithm for smooth convex minimization with Lipschitz continuous gradient. The method is derived via an explicit time discretization of a damped inertial system with vanishing Tikhonov regularization. Under appropriate control of the decay rate of the Tikhonov term, we establish accelerated convergence of the objective values to the minimum together with strong convergence of the iterates to the minimum-norm minimizer. In particular, for polynomial schedules \varepsilon_{k}=k^{-p} with 0<p<2 , we prove strong convergence to the minimum-norm solution while preserving fast objective decay. In the critical case p=2 , we still obtain fast rates for the objective values, while our analysis does not guarantee strong convergence to the minimum-norm minimizer. Furthermore, we provide a thorough theoretical analysis for several choices of Tikhonov schedules. Numerical experiments on synthetic, benchmark, and real datasets illustrate the practical performance of the proposed algorithm. Keywords. Convex optimization, accelerated methods, inertial gradient algorithms, vanishing Tikhonov regularization, minimum-norm selection, continuous-time dynamics and time discretization. AMS 2020 Subject Classification. 90C25, 65K10, 49M37, 37N40. 1 Introduction Let \mathcal{H} be a real Hilbert space endowed with the scalar product \langle\cdot,\cdot\rangle and the associated norm \|x\|=\sqrt{\langle x,x\rangle} . In this paper, we consider the convex minimization problem {\min}\{f(x):x\in\mathcal{H}\}, (1) where f\colon\mathcal{H}\to\mathbb{R} is continuously differentiable, convex, and has L -Lipschitz continuous gradient. We assume that the set of minimizers \mathcal{S}:=\arg\min_{x\in\mathcal{H}}f(x) is nonempty, and we denote by x^{*} its minimum-norm element, that is, the projection of 0 onto \mathcal{S} . When \mathcal{S} is not a singleton, it is well known that, along the classical Tikhonov regularization path, the minimizers of f+\frac{\varepsilon}{2}\|\cdot\|^{2} converge to x^{*} as \varepsilon\to 0^{+} . However, preserving this selection principle in accelerated and inertial schemes, especially in the vanishing regularization regime, is far from automatic. At the continuous-time level, vanishing Tikhonov regularization can enforce convergence to x^{*} , and recent second-order models can in addition yield fast decay of the objective values. At the discrete level, transferring simultaneously acceleration and minimum-norm selection is more delicate: several schemes rely on implicit proximal steps, or introduce multiple regularization terms and intricate coefficient rules. This motivates the search for explicit, Nesterov-type discretizations with a vanishing Tikhonov term that preserve both fast value decay and strong convergence to x^{*} . In this work, we prove that a single vanishing Tikhonov term can enforce minimum-norm selection while preserving accelerated decay of the objective values, within a Nesterov-type inertial scheme. Our aim is to analyze an accelerated first-order inertial gradient algorithm with vanishing Tikhonov regularization and to establish its strong convergence to the minimum-norm minimizer x^{*} . Our approach is based on an explicit time discretization of the Tikhonov regularized inertial gradient system ( TRIGS ) [ 7 , 10 ] : \ddot{x}(t)+\delta\sqrt{\varepsilon(t)}\,\dot{x}(t)+\nabla f(x(t))+\varepsilon(t)\,x(t)=0. (TRIGS) More precisely, our contributions can be summarized as follows: • we derive an explicit Nesterov-type inertial gradient algorithm with a single vanishing Tikhonov regularization term; • we establish fast convergence rates for the objective values and strong convergence of the iterates to the minimum-norm minimizer under general parameter schedules, via a discrete Lyapunov analysis; • we corroborate the theory with numerical experiments on synthetic and real datasets, and we compare our method with NAG and NADTR in terms of solution quality and runtime. From a theoretical viewpoint, this problem is challenging because it requires combining three competing mechanisms: inertia, acceleration, and a vanishing regularization term. While inertia and acceleration tend to amplify oscillations, the Tikhonov term acts as a selection bias toward x^{*} , and calibrating their interplay at the discrete level is delicate. In particular, ensuring both fast decay of the objective values and strong convergence of the iterates requires a careful Lyapunov-based analysis and a precise tuning of the parameters. 1.1 Related works 1.1.1 Continuous-time dynamics First-order dynamical systems with a Tikhonov regularization term whose coefficient vanishes asymptotically have been extensively studied (see, e.g., [ 2 , 1 , 14 , 5 , 15 ] ). In parallel, the second-order setting has also been addressed. The first studies [ 3 , 4 ] by Attouch and Czarnecki considered the coupling of a second-order dynamical system with Tikhonov regularization, \displaystyle\ddot{x}(t)+\gamma\dot{x}(t)+\nabla f(x(t))+\varepsilon(t)x(t)=0. (2) If the Tikhonov regularization term \varepsilon satisfies the slow parametrization condition \int_{0}^{+\infty}\varepsilon(t)\,dt=+\infty , then the unique trajectory of ( 2 ) strongly converges to the minimum-norm element of \arg\min_{\mathcal{H}}f . In contrast, without the Tikhonov regularization term, the convergence remains only weak. Heuristically, including the Tikhonov regularization term introduces a time-dependent strong convexity effect that diminishes over time. To enhance the convergence rate, Attouch, Chbani, and Riahi [ 6 ] explored the following system with asymptotically vanishing damping: \displaystyle\ddot{x}(t)+\dfrac{\alpha}{t}\dot{x}(t)+\nabla f(x(t))+\varepsilon(t)x(t)=0. (3) The system ( 3 ) can be considered as a Tikhonov regularization of the low-resolution generalized ODE modeling the Nesterov accelerated gradient (NAG) method [ 23 ] : \displaystyle\ddot{x}(t)+\dfrac{\alpha}{t}\dot{x}(t)+\nabla f(x(t))=0. As a natural extension of ( 3 ) incorporating Hessian-driven damping, the authors in [ 16 ] investigated a Tikhonov regularization of the high-resolution ODE [ 24 ] defined as follows: \displaystyle\ddot{x}(t)+\dfrac{\alpha}{t}\dot{x}(t)+\beta\nabla^{2}f(x(t))\dot{x}(t)+\nabla f(x(t))+\varepsilon(t)x(t)=0. (4) Both second-order dynamical systems ( 3 ) and ( 4 ) provide fast convergence rates for the objective values. However, the available results on the convergence of the trajectories toward the minimum-norm solution remain limited, as they only guarantee that \liminf_{t\to+\infty}\|x(t)-x^{*}\|=0. The works [ 10 , 7 ] are closely related to ours. The authors introduced new inertial dynamics designed to generate trajectories that converge rapidly to the minimizer of f with minimum norm. They started from the Polyak heavy ball system with friction dynamics for strongly convex functions: \displaystyle\ddot{x}(t)+2\sqrt{\mu}\dot{x}(t)+\nabla f(x(t))=0. (5) Their main idea for handling general convex functions is based on a Tikhonov regularization technique, which naturally leads to a dynamical system governed by the gradient of a strongly convex function \displaystyle\varphi_{t}\colon\mathcal{H}\to\mathbb{R},\quad\varphi_{t}(x):=f(x)+\dfrac{\varepsilon(t)}{2}\|x\|^{2}. Replacing \mu , \nabla f , and the coefficient 2 in ( 5 ) by \varepsilon(t) , \nabla\varphi_{t} , and a positive parameter \delta , respectively, amounts to the aforementioned dynamical system ( TRIGS ). The system ( TRIGS ) is the first second-order dynamical system that guarantees both fast convergence of the values and strong convergence of the trajectories to the minimum-norm solution. The authors in [ 10 , 7 ] obtained these results in the case of a general \varepsilon(\cdot) and gave an in-depth analysis in the particular case \varepsilon(t)=c/t^{2} . The main results in [ 10 , 7 ] are extended and improved in [ 18 ] with the study of the following dynamical system: \displaystyle\ddot{x}(t)+\dfrac{\alpha}{t^{q}}\dot{x}(t)+\nabla f(x(t))+\dfrac{a}{t^{p}}x(t)=0. (6) László showed that the asymptotically vanishing damping coefficient \frac{\alpha}{t^{q}} is correlated with the Tikhonov regularization coefficient \frac{a}{t^{p}} [ 18 ] . He noted that his results significantly extend those in [ 7 ] which only considered the case q=\frac{p}{2} . Moreover, his analysis indicated that the choice p=2q in the dynamical system ( 6 ), which coincides with ( TRIGS ), is not necessarily optimal. By introducing geometric damping driven by the Hessian of the function f into the system ( TRIGS ), the work [ 9 ]