A Conjugate Gradient Method for Nonlinear Programming Problems using Caputo Fractional Gradients

Barsha Shaw a , Md Abu Talhamainuddin Ansary a, · 2025

Read A Conjugate Gradient Method for Nonlinear Programming Problems using Caputo Fractional Gradients by Barsha Shaw a , Md Abu Talhamainuddin Ansary a, (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2512-17634.txt.

Introduction

Barsha Shawa and Md Abu Talhamainuddin Ansary add fractional calculus to a workhorse method for minimizing functions. Conjugate gradient methods walk downhill smartly, reusing the last step's direction to avoid zigzagging. Their variant computes the slope with a Caputo fractional derivative, a slope with memory that still works on kinked, non-smooth surfaces where ordinary slopes fail.

Each step blends the fractional slopes at two straight points, then a careful line search picks the stride. The team proves the walk converges at least linearly under mild assumptions, with a tidy special proof for bowl-shaped functions. In numerical trials, including training a neural network, their method ran faster and steadier than standard rivals. Fractional tools keep finding new footholds in everyday computation.

Catalog record

CREATORBarsha Shaw a , Md Abu Talhamainuddin Ansary a,
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE52 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2512-17634

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

A Conjugate Gradient Method for Nonlinear Programming Problems using Caputo Fractional Gradients Barsha Shaw a , Md Abu Talhamainuddin Ansary a, Abstract The article proposes a Caputo fractional conjugate gradient (CFCG) method for unconstrained optimization problems which is applicable to smooth as well as non-smooth problmes. The proposed method uses a non-adaptive version of the Caputo fractional derivative that provides integer-order derivatives information. A descent direction is obtained using the Caputo fractional gradients of two consecutive iterative points with a parameter ( β \beta ). An inexact line search technique based on Armijo-Wolfe line conditions is used to find a suitable step length. Finally, a descent sequence is generated. The convergence results are derived under mild assumptions that ensuring of convergence is at least linear. Moreover, the convergence of the proposed method for quadratic functions is established through a Tikhonov-regularized formulation that can be interpreted as an extension of the least-squares approach. Finally, some numerical experiments, including neural network applications, are performed to justify that the proposed method achieves faster and more stable performance. 1 Introduction Optimization is a fundamental discipline with applications across nearly every field, where problems are typically formulated as the minimization or maximization of real-valued functions. To address such problems, one of the earliest and most intuitive techniques is the gradient descent method for non-linear optimization problems. This approach iteratively updates the solution by moving in the direction of the negative gradient, which represents the path of the fastest decrease in the function value. This is known as the steepest descent method. However, steepest descent can suffer from slow convergence, especially in ill-conditioned problems. This drawback has led to the development of faster methods such as conjugate gradients (CG) ( [ 1 , 2 ] ). To achieve better convergence with lower computational cost, various forms of the conjugate gradient parameter \beta have been developed. Notable examples include those proposed in [ 3 , 4 , 5 , 6 , 7 ] . These methods generally rely on smooth objective functions. To handle non-smooth functions, proximal techniques have been introduced. The steepest proximal descent, the proximal CG, and the proximal Newton methods are discussed in [ 8 , 9 ] . These methods have many real-life applications, as shown in [ 10 , 11 , 12 ] . An alternative approach for handling non-smooth functions involves the use of non-integer order derivatives, known as fractional derivatives. To capture both the non-smooth behaviour and the information associated with non-integer order differentiation, the concept of fractional calculus becomes useful. Furthermore, due to the integration term involved in the definition of a fractional derivative, it reflects long-memory effects and nonlocal characteristics, which cannot be represented by integer-order derivatives since they depend only on the local behaviour of functions. Additionally, due to the lack of memory effect, steepest descent, conjugate gradient etc method goes in a zigzag way. For this reason, fractional derivative played an advantageous role in research. In the theory of fractional derivatives, among the various existing definitions, the Caputo fractional derivative and the Riemann–Liouville (R–L) derivative are the most widely used. The study of fractional calculus has gained interest due to its applications across many fields such as mechanical and electrical engineering, signal processing, quantum mechanics, bioengineering, biomedicine, financial systems, machine learning, and neural networks ( [ 13 , 14 , 15 , 16 ] ). The R-L fractional derivative {}_{a}D^{\alpha}_{x} and the Caputo fractional derivative {}_{a}^{c}D^{\alpha}_{x} are two standard formulations is present in [ 17 ] . In this article, monotonicity results for both definitions are presented on the interval (0,1) which are later extended to generalized intervals (n,n+1) for n\in\mathbb{N} in [ 18 ] . Several theoretical results for fractional derivatives can also be found in [ 19 ] . A necessary optimality condition by using the Caputo derivative is developed in [ 20 ] . The Taylor–Riemann series and mean value theorem for multivariate functions involving the Caputo derivative are introduced in [ 21 , 22 ] . A gradient descent method based on Caputo derivatives is also proposed there, which relies on a fractional Taylor expansion. Further motivation for employing the Caputo fractional derivative in steepest descent methods can be found in [ 23 ] . However, to reduce high computational complexity and the number of iterations, different strategies can be adopted. For this reason, the present article focuses on a Caputo fractional conjugate gradient method. Main contributions of this article is as follows: • A conjugate gradient method is developed for nonlinear optimization using an fractional order Caputo fractional gradient. • An algorithm is proposed with its convergence results which justified under mild assumptions (Theorems 4.1 - 4.3 ). • The proposed method is compared with existing methods using numerical examples based on neural networks. We have adopted some ideas from [ 26 , 27 , 28 , 29 ] in this regard. The structure of the article is as follows. Section 2 provides the basic definitions and preliminaries required for present developments. In Section 3 , the CFCG method is introduced with different forms of \beta . Section 4 proposed an algorithm for the convergence analysis, specifically for Tikhonov regularisation solutions. Section 5 gives numerical results to show the performance of the method, including examples of neural networks. Section 6 concludes this article with some suggestions on possible directions for future research. 2 Preliminaries We consider the unconstrained optimization problem {(P)}:\quad\min_{x\in\mathbb{R}^{n}}f(x) where f:\mathbb{R}^{n}\rightarrow\mathbb{R} is a continuous function. The steepest descent method is considered a classical numerical technique for solving (P) type problem. In this method, every iterating point x^{k} moves along the descent direction d^{k}=-\nabla f(x^{k}) . However, the linear rate of convergence is a major limitation of this method. This method was further modified in different directions to achieve accelerated convergence. The CG method is one such modification. In this method, the following search direction at x^{k} is d^{k}=\begin{cases}-\nabla f(x^{0}),\quad\quad\quad\quad~\qquad k=0\\ -\nabla f(x^{k})+\beta_{k}\,d^{k-1},\qquad k\geq 1,\par\end{cases} In literature, different choices \beta~(\text{for}~k\geq 1) are used for the CG method to improve convergence speed. In this article, we have considered the following different types of \beta \displaystyle\text{Fletcher-Reever}~(\cite[cite]{[\@@bibref{Number}{FR}{}{}]}):\beta_{k}^{FR}=\frac{\langle\nabla f(x^{k}),\nabla f(x^{k})\rangle}{\langle\nabla f(x^{k-1}),\nabla f(x^{k-1})\rangle} \displaystyle\text{Conjugate-Descent (\cite[cite]{[\@@bibref{Number}{CD}{}{}]})}:\beta_{k}^{CD}=-\frac{\langle\nabla f(x^{k}),\nabla f(x^{k})\rangle}{\langle d^{k-1},\nabla f(x^{k-1})\rangle} \displaystyle\text{Dai-Yuan ($DY$) (\cite[cite]{[\@@bibref{Number}{DY}{}{}]})}:\beta_{k}^{DY}=\frac{\langle\nabla f(x^{k}),\nabla f(x^{k})\rangle}{\langle d^{k-1},\nabla f(x^{k})-\nabla f(x^{k-1})\rangle} \displaystyle\text{Polak-Ribiere-Polyak (\cite[cite]{[\@@bibref{Number}{PRP}{}{}]})}:\beta_{k}^{PRP}=\frac{\langle\nabla f(x^{k}),\nabla f(x^{k})-\nabla f(x^{k-1})\rangle}{\langle\nabla f(x^{k-1}),\nabla f(x^{k-1})\rangle} \displaystyle\text{Hestenes-Stiefel (\cite[cite]{[\@@bibref{Number}{HS}{}{}]})}:\beta_{k}^{HS}=\frac{\langle\nabla f(x^{k}),\nabla f(x^{k})-\nabla f(x^{k-1})\rangle}{\langle d^{k-1},\nabla f(x^{k})-\nabla f(x^{k-1})\rangle}. Throughout this article, the convergence analysis of the proposed CFCG method requires the search direction satisfy sufficient descent condition \nabla f(x^{k})^{\top}d^{k}<-r\,\|d^{k}\|^{2},~r>0. Next, step size \eta_{k} is determined through using the following Armijo-Wolfe conditions, \displaystyle f(x^{k}+\eta_{k}~d^{k})\leq f(x^{k})+c_{1}\eta_{k}~\nabla f(x^{k})^{\top}d^{k}, \displaystyle\nabla f(x^{k}+\eta_{k}~d^{k})^{\top}d^{k}\geq c_{2}\nabla f(x^{k})^{\top}d^{k}, where 0<c_{1}<c_{2}<1 . Finally, the next iterated point is calculated as x^{k+1}=x^{k}+\eta_{k}d^{k} . To accommodate non-smooth behaviour, we work with the class A^{n}[a,b] of functions that are n -times differentiable and whose n+1 -th derivative is absolutely continuous on [a,b] . For a function f\in A^{n}[a,b] , the Caputo fractional derivative of order \alpha\in(n,n+1) at a point x\in[a,b] is defined as {}^{C}_{c}D^{\alpha}_{x}f(x)=\frac{1}{\Gamma(n-\alpha)}\int_{c}^{x}(x-\tau)^{n-\alpha-1}f^{(n)}(\tau)d\tau, A multivariate continuous function, the fractional gradient denoted by {}^{C}_{c}\bar{\nabla}^{\alpha}_{x}f(x) [ 24 ] , is defined as for c\in\mathbb{R}^{n} , {}^{C}_{c}\bar{\nabla}^{\alpha}_{x}f(x)=\Big(~^{C}_{c_{1}}D^{\alpha}_{x_{1}}f(x),~^{C}_{c_{2}}D^{\alpha}_{x_{2}}f(x),\dots,~^{C}_{c_{n}}D^{\alpha}_{x_{n}}f(x)\Big)^{\top}. Although based on multi-variate fractional gradient, a mean value theorem for the Caputo fractional derivative is available (Theorem 15 in [ 21 ] ) from Taylor series. But this

Read the complete text