Decision-Focused Learning via Tangent-Space Projection of Prediction Error

Junhyeong Lee, Sangjin Jin, Yongjae Lee · 2026

Read Decision-Focused Learning via Tangent-Space Projection of Prediction Error by Junhyeong Lee, Sangjin Jin, Yongjae Lee (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-01361.txt.

Introduction

Junhyeong Lee, Sangjin Jin, and Yongjae Lee work on decision-focused learning, where a forecast model is trained to make the final decision good, not just the forecast close. Finding the training signal for that goal usually means grinding through a solver or using stand-in losses, both costly or off target.

The three authors show that, when the active limits of a problem stay stable, the regret signal has a clean geometric form. It is the forecast error cast onto the flat space set by the active limits, scaled by local curvature. In plain terms, parts of the error that cannot change the decision are filtered out. Their method, PEAR, solves one small linear system over the active limits instead of tracing solver steps. On linear benchmarks and a real quadratic task, it gives the best decisions at the lowest cost.

Catalog record

CREATORJunhyeong Lee, Sangjin Jin, Yongjae Lee
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE69 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.01361

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Decision-Focused Learning via Tangent-Space Projection of Prediction Error Junhyeong Lee, Sangjin Jin, Yongjae Lee Abstract Decision-Focused Learning (DFL) trains predictors to improve downstream decision quality, but computing regret gradients typically requires differentiating through solvers or relying on surrogate losses, which can be computationally expensive or deviate from the true objective. We show that, under standard regularity with locally stable active constraints, the regret gradient admits a closed-form geometric characterization, equivalent to the prediction error projected onto the tangent space of active constraints, scaled by local curvature. This reveals that regret gradients can be obtained by filtering decision-irrelevant components from the MSE gradient, providing a simpler and more direct alternative to existing approaches. We propose PEAR ( P rojected E rror A s R egret-gradient), which computes regret gradients via a reduced linear system over active constraints, avoiding differentiation through solver iterations or additional optimization solves. Experiments on LP benchmarks and a real-world QP task show that PEAR achieves the best decision quality among all baselines while being the most computationally efficient, with gains that persist under constraint shifts. Code is available at . † † affiliationnotice: †Corresponding author. 1 Introduction Machine learning is increasingly deployed in decision-making, where a predictive model estimates parameters of an optimization problem and a downstream solver computes the final decision. In these predict-then-optimize settings, reducing the prediction error alone does not necessarily yield good decisions. This has motivated Decision-Focused Learning (DFL) ( 15 ; 17 ) , which trains predictors to improve decision quality, typically measured by regret, the suboptimality incurred by acting on predicted rather than true parameters. Importantly, prediction accuracy and decision quality are not inherently at odds. In the linear-cost setting, minimizing mean squared error can be consistent with reducing expected regret ( 17 ) . This raises a natural question about the structural relationship between prediction error and regret gradients, and how this relationship can inform training. However, this connection remains underexplored. Most prior work has instead focused on procedural questions, namely how to obtain regret gradients. While such methods are valuable for enabling end-to-end learning, existing approaches have notable limitations. Among sensitivity-based differentiable optimization layers, general-purpose methods can trade off efficiency and robustness for broad applicability, and certain constructions require symmetrizing the KKT system, which can lead to ill-conditioned linear solves ( 25 ; 5 ) . A complementary line of work uses surrogate objectives or perturbed estimators to approximate regret gradients. These methods can provide informative learning signals, but they may deviate from the true regret objective, and many are tailored to linear-cost problems, which limits their generality beyond linear objectives ( 28 ; 38 ) . We propose PEAR ( P rojected E rror A s R egret-gradient), a simple projection-based method for computing regret gradients. In contrast to prior work that mainly focuses on how to differentiate through an optimizer or design proxy objectives, we study the structure of regret gradients and their connection to prediction error. Under standard regularity and locally fixed active constraints, we show that the regret gradient admits an explicit geometric form. It is given by projecting the prediction error onto the tangent space of the active constraints under the local curvature metric. This result shows that the regret gradient can be obtained from the MSE gradient by filtering out decision-irrelevant components of the prediction error. Guided by this characterization, PEAR computes regret gradients without differentiating through solver iterations and without relying on surrogate losses. Instead, it evaluates the required projection by solving a compact reduced linear system induced by the active constraints, e.g., via a Schur-complement computation. This avoids backward overhead and improves numerical stability compared to differentiable layers, while remaining faithful to the true regret objective rather than a proxy. PEAR applies to optimization problems with linear or quadratic objectives. Our contributions are as follows: 1. PEAR: a simple projection for regret gradients. We establish an explicit geometric relationship between prediction error and regret gradients in Decision-Focused Learning. This interpretation reveals which error components influence decisions, and PEAR filters out decision-irrelevant directions to improve DFL performance. 2. Efficient and stable computation. PEAR evaluates the projection by solving a compact reduced linear system induced by the active constraints, reducing backward overhead and improving numerical stability compared to differentiable optimization layers. Compared to LP baselines, PEAR provides a deterministic regret gradient without requiring additional solver calls or perturbation-based estimation. 3. Strong empirical performance. On synthetic LP benchmarks and a real-world QP task, PEAR improves regret and training efficiency over other baselines in most settings, and remains strong under noise and constraint perturbations. 2 Related Work Decision-focused learning. Decision-Focused Learning (DFL) trains predictors to improve downstream decision quality, typically measured by regret, in predict-then-optimize pipelines ( 15 ; 42 ; 17 ) . DFL has been studied in a range of applications including healthcare, portfolio optimization, and other optimization-driven decision systems ( 41 ; 12 ; 24 ; 21 ; 16 ) . Recent surveys summarize major methodological themes such as regularization-based approaches, convex surrogate losses, and perturbation-based gradient estimators ( 28 ; 37 ) . A recurring practical challenge is to obtain learning signals that are well aligned with regret while keeping training efficient, especially when training repeatedly interacts with a downstream optimization ( 30 ; 31 ; 7 ) . Figure 1 : The PEAR framework. PEAR provides a direct pipeline for transforming prediction error into a regret gradient via a projection onto the local decision geometry. After identifying the active constraints, PEAR rescales the error by local curvature and projects it onto the feasible tangent space, filtering out decision-irrelevant components. Differentiable optimization layers. A common approach to end-to-end learning is to treat the downstream optimization as a differentiable layer. Early works relied on unrolling iterative optimization steps to compute gradients via the chain rule ( 13 ; 4 ) , while OptNet ( 3 ) introduced implicit differentiation through KKT conditions for quadratic programs. Subsequent work generalized this idea to broader classes of convex programs ( 1 ; 2 ; 20 ; 9 ) . While these methods provide principled gradients, in practice the backward pass depends heavily on the solver implementation. For instance, OptNet requires symmetrized linear systems that can become ill-conditioned ( 5 ) , while CVXPYLayers ( 1 ) routes QPs through cone reformulations, restricting access to specialized solvers and degrading scalability on large sparse instances ( 25 ) . These issues make training expensive and limit solver choices ( 9 ) , motivating recent work on decoupling the forward and backward passes for solver-agnostic sensitivity analysis ( 25 ) . Alternative approaches for regret minimization. Many real-world optimization problems are inherently discrete, leading to piecewise-constant solution maps that yield zero or undefined gradients ( 26 ) . To address uninformative gradients, DFL has largely focused on surrogate objectives and gradient approximations. For linear optimization, SPO+ ( 17 ) is a convex surrogate that upper bounds regret and can be optimized efficiently. A related strategy smooths the optimization problem itself, for example by adding quadratic regularization or using continuous relaxations to obtain meaningful gradients ( 42 ; 27 ) . Alternatively, perturbation-based methods keep the original solver but add random perturbations and use the resulting changes in the

Read the complete text