đťš· \mathbf{\Pi} net: Optimizing hard-constrained neural networks with orthogonal projection layers

Panagiotis D. GrontasETH ZĂĽrich [email protected], Antonio TerpinETH ZĂĽrich [email protected] Efe C. Baltainspire AG & ETH ZĂĽrich [email protected] Raffaello D’AndreaETH ZĂĽrich [email protected] John LygerosETH ZĂĽrich [email protected] · 2025

Read đťš· \mathbf{\Pi} net: Optimizing hard-constrained neural networks with orthogonal projection layers by Panagiotis D. GrontasETH ZĂĽrich [email protected], Antonio TerpinETH ZĂĽrich [email protected] Efe C. Baltainspire AG & ETH ZĂĽrich [email protected] Raffaello D’AndreaETH ZĂĽrich [email protected] John LygerosETH ZĂĽrich [email protected] (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-10480.txt.

Introduction

Grontas, Terpin, Balta, D'Andrea, and Lygeros present a paper from twenty twenty-five on neural networks that always respect hard constraints. Their added output layer projects network answers onto the allowed set. Splitting methods make those projections fast and reliable. Backpropagation runs through an implicit function result. The layer acts as a stand-in solver for families of optimization problems. Single problems solve faster than classic solvers at modest accuracy. Batches of problems solve dramatically faster still.

Training time and solution quality beat prior learning methods by wide margins. Multi-vehicle path planning tests the non-convex side. A graphics-ready package in JAX is released. Feasibility by design is the core promise.

Catalog record

CREATORPanagiotis D. GrontasETH ZĂĽrich [email protected], Antonio TerpinETH ZĂĽrich [email protected] Efe C. Baltainspire AG & ETH ZĂĽrich [email protected] Raffaello D’AndreaETH ZĂĽrich [email protected] John LygerosETH ZĂĽrich [email protected]
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE99 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2508-10480

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

đťš· \mathbf{\Pi} net: Optimizing hard-constrained neural networks with orthogonal projection layers Panagiotis D. GrontasETH ZĂĽrich [email protected], Antonio TerpinETH ZĂĽrich [email protected] Efe C. Baltainspire AG & ETH ZĂĽrich [email protected] Raffaello D’AndreaETH ZĂĽrich [email protected] John LygerosETH ZĂĽrich [email protected] Abstract We introduce an output layer for neural networks that ensures satisfaction of convex constraints. Our approach, Π \Pi net , leverages operator splitting for rapid and reliable projections in the forward pass, and the implicit function theorem for backpropagation. We deploy Π \Pi net as a feasible-by-design optimization proxy for parametric constrained optimization problems and obtain modest-accuracy solutions faster than traditional solvers when solving a single problem, and significantly faster for a batch of problems. We surpass state-of-the-art learning approaches in terms of training time, solution quality, and robustness to hyperparameter tuning, while maintaining similar inference times. Finally, we tackle multi-vehicle motion planning with non-convex trajectory preferences and provide Π \Pi net as a GPU-ready package implemented in JAX with effective tuning heuristics. https://github.com/antonioterpin/pinet pip install pinet-hcnn Figure 1: Illustration of the \Pi net architecture. The infeasible output of the backbone network is projected onto the feasible set through an operator splitting scheme. To train the backbone network, we use the implicit function theorem to backpropagate the loss through the projection layer. 1 Introduction In this work, we deploy neural networks (NNs) to generate feasible-by-design candidate solutions (see Fig. 1 ) for the parametric constrained optimization problem: \underset{y}{\mathrm{minimize}}\quad\varphi(y,\mathrm{x})\quad\mathrm{subject~to}\quad y\in\mathcal{C}(\mathrm{x}), \mathcal{P}(\mathrm{x}) where y\in\mathbb{R}^{d} is the decision variable, \mathrm{x}\in\mathbb{R}^{p} is the context (or parameter) of the problem instance, \varphi:\mathbb{R}^{d}\times\mathbb{R}^{p}\to\mathbb{R} is the objective function, and \mathcal{C}(\mathrm{x})\subseteq\mathbb{R}^{d} is a non-empty, closed, convex set for all \mathrm{x} . We provide a pedagogical example to explain this formulation in Appendix A . Constrained optimization has universal applicability, from safety-critical applications such as the optimal power flow in electrical grids nellikkath2022physics , to logistics and scheduling bengio2021machine , and even biology, where enforcing priors on the solution can enhance its interpretability balcerak2025energy ; terpin2024learning . In many applications, optimization programs are solved repeatedly, given different contexts: in logistics, the demands and forecasts vary baptiste2001constraint ; in model predictive control (MPC) chen2018approximating , the initial conditions; in motion planning, the position of obstacles marcucci2023motion ; in trust-region policy optimization, the advantage function and the trust-region center terpin2022trust . This task often becomes computationally challenging when, for example, y is high-dimensional, \varphi is non-convex, or new solutions are required at a high frequency. Rather than solving each problem instance from scratch, the mapping from contexts to solutions can be learned with NNs . Despite their successes, NNs typically lack inherent mechanisms to ensure satisfaction of explicit constraints, a limitation that motivates a large body of existing work: Soft-constrained NNs. One approach to incorporate constraints in NNs is to include soft penalty terms for constraint violations in the loss function marquez2017imposing . Soft constraints have been used to solve parametric constrained optimization problems tuor2021neuromancer , and partial differential equations through physics-informed NNs erichson2019physics ; raissi2019physics . Despite their ability to handle general constraints, these approaches offer no constraint satisfaction guarantees at inference time. Beside requiring manual tuning of the penalty parameters, which is challenging yet critical for good performance, the use of soft constraints is discouraged for the following reasons. First, the structure of the constraints set can be exploited to design more efficient algorithms; see, e.g., the simplex algorithm dantzig2002linear . Second, treating constraints softly may significantly alter the problem solution regardless of tuning grontas2024operatorsplittingconvexconstrained . Third, certain constrained optimization problems (e.g., linear programs) may not admit a solution at all when constraints are treated softly. Hard-constrained NNs. To circumvent the shortcomings of soft constraints, hard-constrained neural networks (HCNNs) aim to enforce constraints on the NN output by design. The authors in frerix2020homogeneous address linear homogeneous inequality constraints by parameterizing the feasible set. Similarly, RAYEN tordesillas2023rayen enforces various convex constraints by scaling the line segment between infeasible points and a fixed point in the feasible set’s interior. While these methods enjoy rapid inference, they require expensive offline preprocessing and are not directly applicable to constraints that depend on the NN input, i.e., they consider feasible sets \mathcal{C} and not \mathcal{C}(\mathrm{x}) in P ( x ) . Differently, min2024hard proposes a closed-form expression to recover feasibility for polyhedral constraints and employs cvxpylayers agrawal2019differentiable for more general convex sets. In chen2018approximating ; cristian2023end the authors orthogonally project the NN output or intermediate layers using Dykstra’s algorithm boyle1986method , but rely on loop unrolling for backpropagation, which can be prohibitive in terms of memory and computation. Departing from convex sets, DC3 donti2021dc3 introduces an equality completion and inequality correction procedure akin to soft-constrained approaches, but applied during inference. The authors in lastrucci2025enforce impose non-linear equality constraints by recursively linearizing them. Lagrangian and augmented Lagrangian approaches are considered in fioretto2021lagrangian ; park2023self for general non-convex constraints, with drawbacks similar to soft constraints. Recently, LinSATNet wang2023linsatnet has been proposed to impose non-negative linear constraints, which is a restrictive constraint class that renders none of the problems of interest for this work amenable to LinSATNet. This limitation is partially relaxed by GLinSAT zeng2024glinsatgenerallinearsatisfiability . GLinSAT, however, requires bounded constraints, an assumption not satisfied by, e.g., epigraph reformulations ( Stellato_2020 , Appendix A.5-A.7) or in cases where certain variables do not have an immediate physical meaning, e.g., in system level synthesis grontas2022distributed . Implicit layers. Implicit layers embed optimization problems such as quadratic programs (QPs) amos2017optnet ; butler2023efficient , conic programs agrawal2019differentiable , non-linear least-squares pineda2022theseus , or fixed-point equations bai2019deep ; winston2020monotone as NN layers, and apply the implicit function theorem dontchev2009implicit to various measures of optimality: KKT conditions amos2017optnet , homogeneous self-dual embedding agrawal2019differentiable , or fixed-point residuals butler2023efficient ; sun2022alternating , among others blondel2022efficient . ML for optimization. Using constrained NN architectures to learn solution mappings is among many successful efforts to exploit ML techniques to accelerate king2024metric ; sambharya2024learning or replace optimization solvers bertsimas2022online ; zamzam2020learning . These approaches are referred to as amortized optimization , learning to optimize or optimization learning , and the surveys amos2023tutorial ; van2025optimization cover multiple aspects of the topic. Contributions. We propose a novel NN architecture, \Pi net , that generates feasible-by-design solutions for P ( x ) . Given a context \mathrm{x} , we deploy a backbone NN to produce a raw output y_{\text{raw}} that we orthogonally project onto \mathcal{C}(\mathrm{x}) , y=\argmin_{z\in\mathcal{C}(\mathrm{x})}\norm{z - y_{\text{raw}}}^{2}. In particular: • We use an operator splitting scheme to compute the projection in the forward pass, and backpropagate through it via the implicit function theorem. Our work is an instance of implicit layers, but it specializes in projection problems whose simplified structure can be significantly exploited, hence achieving rapid training and improved inference speed. • We implement a hyperparameter tuning and matrix equilibration strategy that boosts \Pi net ’s performance and robustifies it against data scaling. As a result, \Pi net solves challenging benchmarks on which existing methods struggle, and is substantially less sensitive to hyperparameter tuning, as shown through rigorous empirical analysis. In addition, we deploy \Pi net on a real-world application in multi-vehicle motion planning with non-convex trajectory costs. • We provide an efficient and GPU-ready implementation of \Pi net in JAX . We make our code available at https://github.com/antonioterpin/pinet .

Read the complete text