An Inexact General Descent Method with Applications in Differential Equation-Constrained Optimization
Humberto Gimenes Macedo https://orcid.org/0000-0002-0858-6283, Luís Felipe Bueno https://orcid.org/0000-0002-8820-6606 · 2025
Read An Inexact General Descent Method with Applications in Differential Equation-Constrained Optimization by Humberto Gimenes Macedo https://orcid.org/0000-0002-0858-6283, Luís Felipe Bueno https://orcid.org/0000-0002-8820-6606 (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-17581.txt.
Introduction
Optimisation methods usually assume the slope information they receive is exact, yet in many real problems that slope is computed by a solver and is only approximate. Humberto Gimenes Macedo and Luís Felipe Bueno build a descent framework that stays reliable under such inexact gradients. Their motivating case is design work constrained by differential equations, where the gradient comes from an adjoint calculation whose accuracy depends on an inner iterative solver.
The framework allows cheap, coarse gradients early on and sharper ones near the answer, and Macedo and Bueno prove global convergence under two step size rules. They implement it as plain gradient descent and as a quasi-Newton style variant, and test both on two inverse problems. Adaptive accuracy cut the total computing time, and adding curvature information improved efficiency further.
Catalog record
| CREATOR | Humberto Gimenes Macedo https://orcid.org/0000-0002-0858-6283, Luís Felipe Bueno https://orcid.org/0000-0002-8820-6606 |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 79 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2510-17581
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
An Inexact General Descent Method with Applications in Differential Equation-Constrained Optimization Humberto Gimenes Macedo https://orcid.org/0000-0002-0858-6283, Luís Felipe Bueno https://orcid.org/0000-0002-8820-6606 Abstract In many applications, gradient evaluations are inherently approximate, motivating the development of optimization methods that remain reliable under inexact first-order information. A common strategy in this context is adaptive evaluation, whereby coarse gradients are used in early iterations and refined near a minimizer. This is particularly relevant in differential equation–constrained optimization (DECO), where discrete adjoint gradients depend on iterative solvers. Motivated by DECO applications, we propose an inexact general descent framework and establish its global convergence theory under two step-size regimes. For bounded step sizes, the analysis assumes that the error tolerance in the computed gradient is proportional to its norm, whereas for diminishing step sizes, the tolerance sequence is required to be summable. The framework is implemented through inexact gradient descent and an inexact BFGS-like method, whose performance is demonstrated on a second-order ODE inverse problem and a two-dimensional Laplace inverse problem using discrete adjoint gradients with adaptive accuracy. Across these examples, adaptive inexact gradients consistently reduced optimization time relative to fixed tight tolerances, while incorporating curvature information further improved overall efficiency. keywords Inexact optimization, Gradient-based methods, DE-constrained optimization, Discrete adjoint method 1 Introduction Conventionally, convergence analyses of gradient-based algorithms assume the availability of exact information, encompassing both zeroth-order and first-order information. Nevertheless, in various settings, the acquisition of such information is subject to error. These errors can arise, for example, when the objective function lacks an analytic representation, exhibits stochasticity, is affected by finite-precision errors from floating-point arithmetic, or constitutes a smoothed approximation of an inherently non-smooth function. Moreover, errors can also stem from the tolerance-limited accuracy of iterative methods used to compute functional values, gradients, or broader search directions. This phenomenon is especially prevalent in differential equation-constrained optimization (DECO) problems. In this case, the objective function evaluation requires one or more numerical simulations, typically implemented through iterative procedures. Likewise, when the discrete adjoint method is used, computing the gradient normally involves the iterative solution of the adjoint equation. Consequently, in such contexts, both the objective function and its gradient are routinely subject to inexactness. Viewed more broadly, inexactness in numerical analysis arises along two complementary channels, namely (a) inexact solves, where subproblems or linear and nonlinear systems are solved only approximately, and (b) inexact information, where objective, constraint, and/or their gradients evaluations are themselves approximate. Research on nonlinear equations has predominantly focused on channel (a). Foundational work introduced and developed inexact Newton methods to reduce the computational cost associated with solving Newton equations, showing that exact solves are often unnecessary when the current iterate is far from the solution Dembo et al. 1982 ; Eisenstat and Walker 1994 ; Eisenstat and Walker 1996 . This line of study has since been extended to quasi-Newton methods, where Hessian approximations replace the exact Jacobian and the resulting system is solved only approximately. Applications of such techniques to sparse nonlinear systems of equations have been explored Bergamaschi et al. 2001 , and subsequent studies have addressed issues of global convergence and constraints such as non-negativity Birgin et al. 2003 ; Arias et al. 2020 . Recent investigations continue to advance the theory and practice of such inexact methods Arias and Gómez 2023 ; Piétrus et al. 2024 . From an optimization perspective, both channels (a) and (b) have been systematically investigated. Natural extensions of inexact Newton-type techniques to optimization arise from viewing first-order optimality systems as nonlinear equations Martínez 1994 . In unconstrained problems, this leads to inexact Newton and quasi-Newton schemes, while for constrained problems, it gives rise to inexact Sequential Quadratic Programming (SQP) methods. Among these, inexact SQP methods have been extensively studied due to their broad applicability across diverse classes of problems. These methods interpret the resolution of first-order optimality conditions as a sequence of quadratic subproblems solved only approximately, thereby introducing inexactness primarily through channel (a) Leibfritz and Sachs 1999 ; Byrd et al. 2008 ; Curtis et al. 2014 . Inexactness of type (a) also arises when approximately solving subproblems across several optimization frameworks, including higher-order methods Birgin et al. 2017 , interior-point strategies Curtis et al. 2010 , augmented Lagrangian formulations Andreani et al. 2008 , and in the broader class of inexact restoration techniques Bueno and Martínez 2020 . Regarding chanel (b), trust-region and inexact restoration frameworks have been rcently developed to manage accuracy in optimization problems where inexactness is intrinsic, such as finite-sum minimization, in which the objective function is typically approximated via data subsampling to reduce computational cost Curtis et al. 2019 ; Bellavia et al. 2020 ; Bellavia et al. 2023 ; Cao et al. 2024 . Because such approaches enable the scalable training of large-scale AI models, this has become an active and rapidly growing area of research within the optimization community. On the other hand, the investigation of the effect of inexactness in traditional methods, which constitute the foundation of many modern optimization algorithms, dates back to a few decades earlier. The analysis by Polyak 1987 examined how gradient inaccuracies affect the steepest descent method in smooth, convex, unconstrained settings, and Bertsekas and Tsitsiklis 2000 later established convergence guarantees when the error is bounded by the step size times a gradient-dependent term. The accelerated framework of Nesterov 1983 was extended by d’Aspremont 2008 to inexact gradients without loss of optimal complexity. Devolder et al. 2014 formalized the inexact first-order oracle and analyzed primal, dual, and accelerated methods with oracle errors in smooth and convex regimes. The analysis revealed that the accelerated gradient method accumulates errors linearly with the number of iterations, whereas the standard gradient method is not affected by this issue. Building on these findings, Devolder et al. 2013 proposed the intermediate gradient method, which integrates the strengths of both prior approaches. It achieves a convergence rate comparable to the accelerated gradient method while maintaining the robustness of the standard gradient method in the presence of inexact oracles. The scope of the intermediate gradient method was subsequently broadened to address problems involving stochastic oracles in Dvurechensky and Gasnikov 2016 and Ghadimi and Lan 2016 . This analysis was further expanded to include constrained optimization problems Ghadimi et al. 2016 . Khanh et al. 2024a developed an inexact reduced gradient method for nonconvex, continuously differentiable problems, both with and without Lipschitz continuous gradients, establishing convergence under various step-size strategies. This research direction was further explored in Khanh et al. 2024b , where a new inexact gradient method for smooth functions was introduced. Inexact trust-region and multilevel SQP algorithms have also been developed for DECO applications, where both the state and adjoint equations, as well as the quadratic subproblems, are solved only approximately, with adaptive mechanisms controlling inexactness and discretization accuracy Heinkenschloss and Ridzal 2008 ; Ziems and Ulbrich 2011 . In the same context, Brown and Nadarajah 2018a ; Brown and Nadarajah 2018b ; Brown and Nadarajah 2022 analyzed how inexactness in discrete adjoint computations impacts gradient methods. They argue that when the state and adjoint equations are solved with an adaptive tolerance proportional to the gradient norm, the inexact gradient method achieves the same convergence rate as its exact counterpart. In these works there is a connection