Preconditioning Natural and Second Order Gradient Descent in Quantum Optimization: A Performance Benchmark

Théo Lisart-Liebermann, Arcesio Castañeda Medina, Fraunhofer ITWM, 67655 Kaiserslautern, Germany · 2025

Read Preconditioning Natural and Second Order Gradient Descent in Quantum Optimization: A Performance Benchmark by Théo Lisart-Liebermann, Arcesio Castañeda Medina, Fraunhofer ITWM, 67655 Kaiserslautern, Germany (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-16518.txt.

Introduction

Théo Lisart-Liebermann and Arcesio Castañeda Medina benchmark ways to tune quantum circuits in a 2025 study. Tuning such circuits is hard: the goal surface is bumpy, gradient reads are noisy, and flat zones can stall learning. Methods that use curvature can converge faster than plain gradient steps, but they cost more per step and react badly to noise.

The team compared three families of optimizers on made-up MaxCut problems run through a shallow quantum routine. A tweaked quasi-Newton method that damps its updates against noisy steps gave the best results for this task. The study helps users weigh speed, cost, and noise tolerance when they pick an optimizer for near-term quantum work.

Catalog record

CREATORThéo Lisart-Liebermann, Arcesio Castañeda Medina, Fraunhofer ITWM, 67655 Kaiserslautern, Germany
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE68 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2504-16518

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Preconditioning Natural and Second Order Gradient Descent in Quantum Optimization: A Performance Benchmark Théo Lisart-Liebermann, Arcesio Castañeda Medina, Fraunhofer ITWM, 67655 Kaiserslautern, Germany Abstract The optimization of parametric quantum circuits is technically hindered by three major obstacles: the non-convex nature of the objective function, noisy gradient evaluations, and the presence of barren plateaus. As a result, the selection of classical optimizer becomes a critical factor in assessing and exploiting quantum-classical applications. One promising approach to tackle these challenges involves incorporating curvature information into the parameter update. The most prominent methods in this field are quasi-Newton and quantum natural gradient methods, which can facilitate faster convergence compared to first-order approaches. Second order methods however exhibit a significant trade-off between computational cost and accuracy, as well as heightened sensitivity to noise. This study evaluates the performance of three families of optimizers on synthetically generated MaxCut problems on a shallow QAOA algorithm. To address noise sensitivity and iteration cost, we demonstrate that incorporating secant-penalization in the BFGS update rule (SP-BFGS) yields improved outcomes for QAOA optimization problems, introducing a novel approach to stabilizing BFGS updates against gradient noise. 1 Introduction Variational Quantum Algorithms (VQAs) are considered the main candidate for near-term quantum advantage [ 1 , 2 , 3 , 4 , 5 ] . These algorithms employ an iterative process that leverages the strengths of both quantum and classical computing. The approach involves evaluating an objective function using a parameterized quantum circuit on quantum hardware, and then utilizing a classical optimization routine to refine the results. This hybrid strategy is particularly valuable when the objective function can be efficiently computed on quantum hardware, potentially offering significant performance advantages across various applications: quantum chemistry [ 6 , 7 , 8 , 9 ] , combinatorial problems [ 10 , 11 , 12 , 13 ] , simulation [ 14 , 15 , 16 ] , control [ 17 , 18 , 19 ] , quantum machine learning (QML) [ 20 , 21 ] , numerical methods subroutines [ 22 ] . The performance of these applications is heavily influenced by the classical routine’s ability to identify a satisfactory approximation of the global optimum. This capability depends on the selection of an optimal optimization algorithm and the underlying optimization landscape associated with the objective function, which can be compromised by various factors. In particular, the so-called barren plateaus (BP) lead to entire regions of vanishing gradients as the result of sources such as noise [ 23 ] , type of cost-function [ 24 ] , and ansatz design and presence of entanglement [ 25 , 26 ] . Although recent research has shown that the origin of BP from the ansatz structure can be avoided for a large class of problems [ 27 , 28 , 29 , 30 ] , poor gradient quality remains a practical concern in many cases [ 31 ] . Optimization of VQA models in realistic settings is hampered by the inherent non-convexity of their objective functions [ 5 , 32 , 33 ] and noisy gradient estimates [ 34 ] . This dual challenge undermines theoretical guarantees of convergence for traditional optimization methods [ 35 , 36 , 37 ] , and the classical optimizer must be tailored to the specific problem. Success in optimizing parametric quantum circuits relies on tradeoffs between minimizing quantum hadware calls, number of iterations and performance in noisy function calls. Previous efforts focused on benchmarking gradient-less (zero-order order) and first order methods. Thus, for example, authors in [ 38 ] approach noisy objective function evaluation using stochastic methods, in particular the Simultaneous Perturbation Stochastic Approximation (SPSA) [ 39 ] and second order extention [ 40 ] . On the other hand, in [ 41 ] an extensive study of SciPy [ 42 ] implementations of popular optimizers is provided, covering stochastic gradient methods, zero-order and first order methods. This last work extends on [ 43 ] , which focused on establishing best practice for gradient-free methods in controlled noisy emulators. The latter argues that specifically tuned optimizers are crutial and will be needed even in the presence of usable error correction schemes, justifying the need for improvements using hyper-parameter fine tuning using more advanced surrogate models [ 44 , 45 ] . Within those studies, particular attention was brought to the definition of relevant cost models designed to balance empirical efficiency (or direct runtime cost) and solution accuracies relevant for the specific needs of quantum computing. A rich literature of methods from classical optimization has emerged in the context of VQA-based quantum optimization [ 46 , 47 , 48 , 49 ] . For reference, we can list the Nelder-Mead simplex algorithm [ 50 ] , SPSA [ 51 ] , collective and swarm optimization (ant-based, particle-swarm), Bayesian optimization, gradient-based reinforcement learning, Powell’s method, Conjugate Gradient, Constrained Optimization by Linear Approximation (COPYLA), Sequential Least Squares Programming (SLSQP), Broyden–Fletcher–Goldfarb–Shanno (BFGS), and Byrd-Omojokun Trust Region Sequential Quadratic Programming (trust-constr) [ 52 ] . Another family of promising methods comes from the field of machine learning, which has seen significant success over the last two decades. Namely gradient-based methods such as Stochastic Gradient Descent (SGD), Adaptive Gradient Algorithm (AdaGrad) [ 53 ] , Root Mean Square Propagation (RMSProp) [ 54 ] , Follow the Regularized Leader (FTRL) [ 55 ] , Adam, Adamax, Nadam, and AMSGrad [ 56 , 57 , 58 ] . Machine-learning centric methods have been ranked for quantum optimization problems in [ 41 ] . Systematic testing of cutting-edge quasi-Newton methods has not been as thoroughly examined as zero-order and first-order methods. Benchmarking different variants of these methods, using distinct Hessian matrix approximations and alternative shot noise mitigation strategies, can provide better alternatives for VQAs. Thus, for example, although the Davidon-Fletcher-Powell (DFP) method [ 59 ] , which predates BFGS and its limited memory variant [ 60 ] , has been shown to be less accurate and stable than its successor, fine-tuned DFP could potentially lead to faster convergence due to its more aggressive Hessian approximation. Other methods, like the The Symmetric Rank One (SR1) method [ 61 ] , offer a flexible Hessian matrix approximation, particularly in situations where well-posedness cannot be guaranteed. Another overlooked approach is the quasi-Newton update based on the Conjugate Gradient (NCG) Hessian approximation [ 62 ] , which inherits the advantages of CG for quasi-Newton updates while offering improved scaling and memory usage, although tends to perform poorly in non-convex geometries. An alternative strategy for leveraging second-order information at key points in parameter space involves applying a contemporary approach that incorporates a gradient quality metric when updating the BFGS algorithm (SP-BFGS) [ 63 ] . By penalizing the update, this method has a stabilizing effect on the Hessian approximation, enhancing overall robustness and dynamical properties. The technique seamlessly interpolates between first- and second-order methods, thereby preventing accumulation of errors in the Hessian approximation, and generalizes BFGS updates to noisy function evaluations. Second order methods scales the step size of the Newton update according to the Hessian matrix information. It is known however that local gradients evaluations are not the best possible update direction [ 64 , 65 ] . Natural Gradient Descent (NGD) use the Fisher

Read the complete text