Convolutional optimization with convex kernel and power lift
Zhipeng Lu · 2025
Read Convolutional optimization with convex kernel and power lift by Zhipeng Lu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-22135.txt.
Introduction
Zhipeng Lu sets out a new optimization frame built on convolution with convex kernels, in work from 2025. Most global search methods lean on chance based models that give no firm promise. The aim here is a more rule bound model for finding the global best point of a general function.
A power lift step joins the convex kernel convolution at the core of the frame. Early numeric tests check how well derived algorithms work in practice. Results so far are limited and early, and the author says so plainly. The note serves as a founding sketch meant to draw more work. Readers in optimization get the core idea and first proof of concept runs.
Catalog record
| CREATOR | Zhipeng Lu |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 56 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2503-22135
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Convolutional optimization with convex kernel and power lift Zhipeng Lu Abstract 1. Introduction In this work, we focus on establishing the foundational paradigm of a novel optimization theory based on convolution with convex kernels. Our goal is to devise a morally deterministic model of locating the global optima of an arbitrary function, which is distinguished from most commonly used statistical models. Limited preliminary numerical results are provided to test the efficiency of some specific algorithms derived from our paradigm, which we hope to stimulate further practical interest. 2. Theoretical basics Our paradigms of optimization theory are motivated by the following basic mathematical facts. 2.1. Convolution preserving convexity Our optimization paradigm is based on a simple observation on the preservation of convexity by convolution. We start with the univariate case. Lemma 1 . Suppose f(x)\geq 0 is continuous and g(x) is convex. Then their convolution g\ast f(x) is convex. Proof. For any \theta_{1},\theta_{2}\in\mathbb{R} , write \displaystyle g\ast f(\theta_{1})+g\ast f(\theta_{2})-2g\ast f\left(\frac{\theta_{1}+\theta_{2}}{2}\right) \displaystyle= \displaystyle\int_{\mathbb{R}}g(\theta_{1}-t)f(t)dt+\int_{\mathbb{R}}g(\theta_{2}-t)f(t)dt-2\int_{\mathbb{R}}g\left(\frac{\theta_{1}+\theta_{2}}{2}-t\right)f(t)dt \displaystyle= \displaystyle\int_{\mathbb{R}}\left(g(\theta_{1}-t)+g(\theta_{2}-t)-2g\left(\frac{(\theta_{1}-t)+(\theta_{2}-t)}{2}\right)\right)f(t)dt. If g is convex, g(\theta_{1}-t)+g(\theta_{2}-t)-2g\left(\frac{(\theta_{1}-t)+(\theta_{2}-t)}{2}\right)\geq 0 so that the last integral is nonnegative provided f\geq 0 . Thus, g\ast f is convex. ∎ When g has a continuous second derivative, the proof becomes more concise notionally. Note that (g\ast f)^{\prime}=g^{\prime}\ast f and (g\ast f)^{\prime\prime}=g^{\prime\prime}\ast f . Hence, if g is convex with a continuous second derivative, then g^{\prime\prime}\geq 0 so that g^{\prime\prime}\ast f\geq 0 given that f\geq 0 . That is, the convolution g\ast f is convex. . In higher dimensions, similarly by linearity \frac{\partial(g\ast f)}{\partial x_{i}}=\frac{\partial g}{\partial x_{i}}\ast f and \frac{\partial^{2}g\ast f}{\partial x_{i}\partial x_{j}}=\frac{\partial^{2}g}{\partial x_{i}\partial x_{j}}\ast f . Also note that convexity in \mathbb{R}^{n} is equivalent to convexity along any line, i.e. g(x_{0}+tv) is convex in t for any x_{0},v\in\mathbb{R}^{n} . In other words, since \frac{d^{2}g(x_{0}+tv)}{dt^{2}}=v^{T}(\nabla^{2}g)v with \nabla^{2}g the Hessian of g , convexity of g means \nabla^{2}g is positive semi-definite. Hence again by linearity, for any x_{0},v\in\mathbb{R}^{n} , \frac{d^{2}(g(x_{0}+tv)\ast f)}{dt^{2}}=(v^{T}\nabla^{2}gv)\ast f, which is non-negative if f\geq 0 and g is convex. Thus, without further digestion, for general convex functions in higher dimensions, we also have Corollary 1 . Suppose f\geq 0 is continuous and g is convex on \mathbb{R}^{n} for any n\geq 1 . Then g\ast f is convex. In practice, if f is not nonnegative, we may increase it by a large enough constant C>0 so that f+C\geq 0 ; the continuity of f is necessary as in most algorithms in which discrete sampling is utilized following suitable probability distributions. As we will see, whether f has a unique maximum or not is crucial in our paradigm especially in the process of “integral concentration”. 2.2. Critical point approximation The next key principle of our paradigm is to approach the maximum point of f by those of g_{n}\ast f with a series of convex functions g_{n} . The following observation on critical points of univariate convolutions is key to our construction. Lemma 2 . Suppose g has a continuous first derivative |g^{\prime}(x)|\leq 1 and f\geq 0 . If there exist \delta,\epsilon>0 such that |g^{\prime}(x)|>\frac{1}{2} for |x|\geq\delta and \int_{|x-a|\leq\epsilon}fdx\geq\frac{2}{3}\int_{\mathbb{R}}fdx for some a , then any potential stationary point of g\ast f lies in (a-\delta-\epsilon,a+\delta+\epsilon) . Proof. We decompose the integral of g\ast f into two parts ( B_{\epsilon}(0)=(-\epsilon,\epsilon) ): \displaystyle(g\ast f)^{\prime}(\theta) \displaystyle=\int_{\mathbb{R}}g^{\prime}(\theta-a-x)f(a+x)dx \displaystyle=\int_{B_{\epsilon}(0)}g^{\prime}(\theta-a-x)f(a+x)dx+\int_{\mathbb{R}\smallsetminus B_{\epsilon}(0)}g^{\prime}(\theta-a-x)f(a+x)dx. The second part is easily estimated as \left|\int_{\mathbb{R}\smallsetminus B_{\epsilon}(0)}g^{\prime}(\theta-a-x)f(a+x)dx\right|\leq\int_{\mathbb{R}\smallsetminus B_{\epsilon}(0)}f(a+x)dx<\frac{1}{3}\int_{\mathbb{R}}f(x)dx. If |\theta-a|\geq\delta+\epsilon , then |\theta-a-x|\geq\delta,\forall x\in B_{\epsilon}(0) so that \int_{B_{\epsilon}(0)}g^{\prime}(\theta-a-x)f(a+x)dx\begin{cases}>\frac{1}{2}\int_{B_{\epsilon}(0)}f(a+x)dx>\frac{1}{3}\int_{\mathbb{R}}f(x)dx,\ \theta-a\geq\delta+\epsilon,\\ <-\frac{1}{2}\int_{B_{\epsilon}(0)}f(a+x)dx<-\frac{1}{3}\int_{\mathbb{R}}f(x)dx,a-\theta\leq\delta+\epsilon.\end{cases} Thus if |\theta-a|\geq\delta+\epsilon , (g\ast f)^{\prime}(\theta)\neq 0 . ∎ In higher dimensions, since all the first partial derivatives vanish at a critical point, if the conditions of Lemma 2 suit for \frac{\partial g}{\partial x_{i}} and f , the same conclusion holds. Corollary 2 . Suppose g has continuous partial derivatives |\partial g/\partial x_{i}|\leq 1 and f\geq 0 is continuous. If for each i , there exists \delta_{i}>0 such that |\partial g/\partial x_{i}|>\frac{1}{2} for |x_{i}|\geq\delta_{i} and \int_{B_{\epsilon}(a)}fdx\geq\frac{2}{3}\int_{\mathbb{R}^{n}}fdx for some \epsilon>0 and a=(a_{1},\dots,a_{n})\in\mathbb{R}^{n} , then any potential stationary point \theta=(\theta_{1},\dots,\theta_{n}) of g\ast f satisfies |\theta_{i}-a_{i}|<\delta_{i}+\epsilon for each i . 2.3. Construction of convex kernel We begin with the one-dimensional case. As indicated by Lemma 2 , the convolution kernel g should behave like a “quasilinear” function. It is also supposed to be convex to avoid local optima traps as conceived in the section 2.1 . Choices are still abundant in these regards. However, in practice, since computing convolution is already expensive, our choice had better be fixated on “low degree” quasilinear functions. This leads us to design g with only linear and quadratic fractions. To make it even more economical, let g be even with linear fractions |x\pm\delta|+h outside |x|\leq\delta for some border hight h ; inside |x|\leq\delta , we use a quadratic fraction ax^{2}+b to smooth out the boundary. On the border, continuity requires the latter to reach height h , while differentiability requires 2ax=\pm 1 . Then we have the simplest system (1) \begin{cases}a\delta^{2}+b=h\\ 2a\delta=1,\end{cases} which is solved by a=\frac{1}{2\delta} and b=h-\frac{\delta}{2} . Thus, we simply define for any \delta>0 (2) g(x)=\begin{cases}x-\delta+h,\quad x>\delta,\\ \frac{1}{2\delta}x^{2}-\frac{\delta}{2}+h,\quad-\delta\leq x\leq\delta,\\ -x+\delta+h,\quad x<-\delta.\end{cases} Note that g behaves like a smoothed absolute value function, whose convexity is self-explained. The one-dimensional construction is instructive enough for any higher dimension: meld a cone with a paraboloid along a sphere! Without further digestion, we spout the definition for \mathbb{R}^{n} as (3) g(x)=\begin{cases}\|x\|-\delta+h=\sqrt{x_{1}^{2}+\cdots+x_{n}^{2}}-\delta+h,\quad\|x\|>\delta,\\ \frac{1}{2\delta}\|x\|^{2}-\frac{\delta}{2}+h=\frac{1}{2\delta}(x_{1}^{2}+\cdots+x_{n}^{2})-\frac{\delta}{2}+h,\quad\|x\|\leq\delta.\end{cases} If necessary, one may define g as melds of variant cones with general paraboloids along suitable ellipsoids. However, such variant seems not profitable for our paradigm. 2.4. Integral concentration Again, as indicated by Lemma 2 , the last key point of our paradigm is to rescale f so that its integral is concentrated around its optimum. Our simplest idea is to increase the gap between smaller and larger values while not shifting the variable, which suggests the use of composition functions \varphi\circ f with monotonically increasing functions \varphi . Moreover, with respect to the direction of inequality required by Lemma 2 , we had better use convex \varphi , resorting to Jensen’s inequality: \frac{1}{b-a}\int_{a}^{b}\varphi(f(x))dx\geq\varphi\left(\frac{1}{b-a}\int_{a}^{b}fdx\right). Concerned with computational cost, a convenient choice is \varphi(x)=x^{N} (for large N ). In this case, we may derive an efficient bound for the super parameter N as follows. Lemma 3 . Suppose f\geq 0 is continuous with a unique maximum at a and compactly supported in \mathbb{R}^{n} . Then for any \epsilon>0 such that the average of f in B_{\epsilon}(a) is strictly larger than the essential supremum of f outside B_{\epsilon}(a) , say 1+\rho_{f}(\epsilon) multiple of the latter, there exists N=O\left(\frac{n}{\rho_{f}(\epsilon)}\ln\left(\frac{1}{\epsilon}\right)\right) such that \int_{B_{\epsilon}(a)}\varphi(f(x))dx\geq\frac{2}{3}\int_{\mathbb{R}^{n}}\varphi(f(x))dx Proof. Suppose f is supported on a compact set C\subset\mathbb{R}^{n} . Then by Jensen’s inequality \frac{\int_{B_{\epsilon}(a)}f^{N}dx}{\int_{C\smallsetminus B_{\epsilon}(a)}f^{N}dx}\geq\frac{|B_{\epsilon}(a)|\left(\frac{1}{|B_{\epsilon}(a)|}\int_{B_{\epsilon}(a)}fdx\right)^{N}}{\int_{C\smallsetminus B_{\epsilon}(a)}f^{N}dx}=|B_{\epsilon}(a)|\left(\frac{\frac{1}{|B_{\epsilon}(a)|}\int_{B_{\epsilon}(a)}fdx}{\left(\int_{C\smallsetminus B_{\epsilon}(a)}f^{N}dx\right)^{1/N}}\right)^{N}. Note that \|f\|_{N^{\prime}}\leq\|f\|_{N}\leq|C|^{\frac{1}{N}-\frac{1}{N^{\prime}}}\|f\|_{N^{\prime}} for any 1<N<N^{\prime} , and the denominator \|f|_{C\smallsetminus B_{\epsilon}(a)}\|_{N} tends to \|f|_{C\smallsetminus B_{\epsilon}(a)}\|_{\infty}\leq(1+\rho_{f}(\epsilon))\frac{1}{|B_{\epsilon}(a)|}\int_{B_{\epsilon}(a)}fdx by assumption. In addition, |B_{\epsilon}(a)|=O(\epsilon^{n}) . Hence, to bound the above ratio away from 0, we need ( |C)| is supposed to be negligible) N=O\left(\frac{\ln(1/\epsilon^{n})}{\ln(1+\rho(\epsilon)}\right)=O\left(\frac{n}{\rho_{f}(\epsilon)}\ln\left(\frac{1}{\epsilon}\right)\right). ∎ We may also use the exponential function e^{Nx} to rescale target functions as how softmax works, but it seems easy to blow up the integral calculation. Compared to our naive baseline setup with simple power functions, one may choose better rescaling functions suitable for the value distributions of special target functions. 2.5. Restriction of convex functions This section provides a slightly deeper insight into convexity, which inspires