Constrained Neural Parameterization for Optimization in Function Spaces
Michael Hintermüller 1, Jianfeng Ning 2 · 2026
Read Constrained Neural Parameterization for Optimization in Function Spaces by Michael Hintermüller 1, Jianfeng Ning 2 (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-00855.txt.
Introduction
Hintermüller and Ning turn hard constraints into smooth freedom in this 2026 maths paper. Many optimization tasks live in function spaces, where the unknown is a whole function. Constraints fence the search: the answer must stay in an allowed set. Penalty terms and multipliers are the usual, clumsy tools.
Their scheme builds neural parameterizations whose outputs never leave the allowed set. The maps are smooth, and their reach grows dense in the set over time. With such a map, the fenced problem becomes a smooth open one in parameter space. Gradient methods then run with no penalties and no multipliers. They give builds for polyhedral sets in Hilbert spaces and for point-wise constraints. For some PDE constraints with separable form, they give an exact reduced neural form. Tests show the methods work well in practice, they report.
Catalog record
| CREATOR | Michael Hintermüller 1, Jianfeng Ning 2 |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 90 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2606.00855
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Constrained Neural Parameterization for Optimization in Function Spaces Michael Hintermüller 1, Jianfeng Ning 2 Abstract We propose constrained neural parameterization schemes for several classes of constraints arising in optimization problems in function spaces. This is achieved by constructing smooth neural parameterizations whose image lies entirely in the admissible set while remaining asymptotically dense. In this way, the original constrained optimization problem is transformed into a smooth unconstrained problem in parameter space, enabling efficient gradient-based optimization without penalty parameters or Lagrange multipliers. We develop geometric constructions for polyhedral constraint sets in Hilbert spaces, propose smooth neural architectures for some pointwise constraints, and introduce an exact reduced neural formulation for PDE constraints that admit a separable structure. Numerical experiments demonstrate the effectiveness of the proposed methods. 1 1 footnotetext: Email: [email protected] 2 2 footnotetext: Email: [email protected] Keywords : constrained neural parameterization, PDE-constrained optimization, polyhedral constraints, state constraints, PINNs 1 Introduction Constrained optimization in function spaces plays a central role in many areas of applied mathematics, including optimal control, inverse problems, and shape optimization. In these applications one often seeks to minimize a functional over an admissible set defined by structural conditions such as pointwise inequalities, integral constraints, or partial differential equations (PDEs). Such constraints are not merely technical requirements but typically reflect intrinsic modeling principles: pointwise bounds encode physical limitations, integral constraints represent conservation laws or resource restrictions, complementarity conditions arise from equilibrium phenomena, and PDE constraints describe governing physical laws. Designing numerical methods that are both computationally efficient and capable of reliably enforcing such constraints therefore is of paramount importance. Consider the constrained optimization problem in a Banach space X equipped with the norm \|\cdot\|_{X} : \min_{x\in X}J(x)\quad\text{subject to}\quad x\in\mathcal{K}\subset X, (1.1) where J:X\to\mathbb{R} is the objective functional and \mathcal{K} denotes an admissible set. In this work we propose a class of constrained neural parameterization (CNP) methods that enforce feasibility by construction for some common constraints. The central idea, advocated here, is to introduce a smooth parametric mapping \mathcal{T}_{M}:\mathbb{R}^{M}\to X, (1.2) whose image is contained entirely in \mathcal{K} . The mapping is designed so that its range becomes dense in \mathcal{K} as the parameter dimension increases. This means in particular that \forall\,x\in\mathcal{K} , there exists a sequence \{\theta_{M}\} such that \|\mathcal{T}_{M}(\theta_{M})-x\|_{X}\rightarrow 0,\quad\text{as }M\rightarrow\infty. (1.3) Consequently, the constrained problem ( 1.1 ) can be approximated by the unconstrained finite-dimensional optimization problem for sufficiently large M : \min_{\theta\in\mathbb{R}^{M}}J(\mathcal{T}_{M}(\theta)), (1.4) whose candidate solutions automatically satisfy the constraints. Gradient-based optimization can therefore be performed directly in parameter space without the need for penalty parameters, projections, or multiplier updates. Thus, feasibility is enforced by construction rather than by external constraint-handling mechanisms. The mathematical foundations of constrained optimization are rooted in Lagrangian duality, variational analysis, and Karush–Kuhn–Tucker (KKT) theory. Classical references such as [ 3 , 35 ] provide comprehensive treatments of optimality conditions, constraint qualifications, and duality frameworks. In convex settings, interior-point and barrier methods [ 40 , 14 ] offer algorithms with strong theoretical guarantees and approximate the original constrained problem by a sequence of unconstrained problems. Their numerical realization, however, requires careful tuning of barrier parameters and related algorithmic choices. For nonlinear problems, sequential quadratic programming, augmented Lagrangian methods, and exact penalty formulations [ 8 , 7 , 12 ] are widely used and exhibit superlinear local convergence under suitable regularity assumptions. When these methods are extended to infinite-dimensional settings, additional analytical and numerical challenges arise. In particular, classical numerical approaches often enforce constraints through penalty terms, projection operators, or active-set strategies [ 31 , 15 ] . Penalty-based formulations require careful tuning of weights and may suffer from ill-conditioning as penalty parameters increase [ 6 ] . PDE-constrained optimization has been systematically studied over the past decades; see, for instance, [ 22 , 39 , 19 ] . Classical solution strategies are typically based on reduced formulations through the control-to-state mapping, adjoint equations for gradient computation, and semismooth Newton or active-set techniques for handling constraints. These approaches form the backbone of efficient gradient-based algorithms in both theory and practice. In practical applications, PDE-constrained optimization problems often involve some control or state constraints. Control constraints, such as box restrictions on distributed controls, are relatively well understood and can often be treated using projection formulas or primal-dual active-set strategies within the KKT framework. In contrast, state constraints introduce substantially greater analytical and numerical difficulties. For elliptic and parabolic problems with pointwise state constraints, the associated Lagrange multipliers may belong to spaces of Radon measures or exhibit low Sobolev regularity rather than lie in L^{p}(\Omega) [ 17 , 39 ] . Furthermore, classical constraint qualifications may fail in infinite-dimensional settings, and strict complementarity may break down, especially in problems involving complementarity-type structures or biactive sets of positive measure [ 16 ] . These difficulties often lead to instability in multiplier-based algorithms and require sophisticated regularization, interior-point, or active-set strategies. In recent years, neural network methods have emerged as flexible, mesh-free approaches for solving PDEs and related inverse problems [ 34 , 44 , 33 , 28 ] . Several neural frameworks have also been developed for PDE-constrained optimal control problems. One class of approaches incorporates the PDE residual as a penalty term in the objective function, as in physics-informed neural networks (PINNs) [ 29 , 32 ] . Another class of methods relies on neural approximations of the KKT system. For example, the adjoint-oriented neural network (AONN) proposed in [ 42 ] alternately updates the state, adjoint, and control variables. The AOMAD method [ 43 ] combines adjoint techniques with meta-learning strategies to efficiently handle parametric optimal control problems. The OSNN method [ 10 ] approximates the state and adjoint variables in elliptic optimal control problems using separate networks and solves the resulting reduced optimality system using PINN techniques. In [ 36 ] , the alternating direction method of multipliers was integrated with PINNs to address nonsmooth PDE-constrained optimization problems. Despite their flexibility, enforcing constraints robustly within neural frameworks remains challenging. Existing KKT-based neural approaches mainly address control constraints and become difficult to apply to state-constrained problems when the associated multipliers exhibit low regularity. Moreover, many existing neural approaches enforce the PDE constraint through residual minimization, so that the resulting numerical state and control generally do not satisfy the PDE relation exactly. Another limitation of PINN-based neural approaches for PDE-constrained optimization is that they require sufficient regularity assumptions on the state, which may not be satisfied in many situations, such as irregular domains. We also mention some operator learning methods developed for PDE-constrained optimization where the learned control-to-state mapping is employed to search for optimal control; see [ 26 , 13 ] . Beyond PDE solvers, considerable effort has been devoted to designing neural network architectures that preserve structural properties of the target functions. Examples include divergence-free constructions for incompressible flows via stream functions [ 34 ] , hard enforcement of Dirichlet and periodic boundary conditions [ 29 ] , invertible neural networks [ 4 ] , and input-convex neural networks for convexity preservation [ 2 ] . These developments demonstrate that embedding constraints directly into the architecture, rather than enforcing them through penalties, can significantly improve numerical robustness and interpretability. While existing structure-preserving architectures typically enforce specific functional properties, much less attention has been devoted