Efficient End-to-End Learning for Decision-Making: A Meta-Optimization Approach
Rares Cristian, Pavithra Harsha, Georgia Perakis, Brian Quanz · 2025
Read Efficient End-to-End Learning for Decision-Making: A Meta-Optimization Approach by Rares Cristian, Pavithra Harsha, Georgia Perakis, Brian Quanz (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-11360.txt.
Introduction
Rares Cristian and co-authors speed up end-to-end learning, where a prediction model is trained with an eye on the choice that its forecast will drive. Such models beat the split route, which trains on forecast error alone, but each step of training needs a costly optimization solve.
Their method learns a fast stand-in solver. A neural network gives near best answers while a projection step keeps the answers within the rules. The authors prove fast approach to the optimum, bounds on the error, and bounds on how well it works on new cases. The stand-in is faster and scales better than past methods.
Catalog record
| CREATOR | Rares Cristian, Pavithra Harsha, Georgia Perakis, Brian Quanz |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 99 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-11360
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Efficient End-to-End Learning for Decision-Making: A Meta-Optimization Approach Rares Cristian, Pavithra Harsha, Georgia Perakis, Brian Quanz Abstract End-to-end learning has become a widely applicable and studied problem in training predictive ML models in order to be aware of their impact on downstream decision-making problems. These end-to-end models often outperform traditional methods that separate the prediction from the optimization steps and only myopically focus on prediction error. However, the computational complexity of end-to-end frameworks poses a significant challenge, particularly for large-scale problems. This is because while training an ML model using gradient descent, each time we need to compute a gradient we must solve an expensive optimization problem. We present a meta-optimization method that learns efficient algorithms to approximate optimization problems, dramatically reducing computational overhead of solving the decision problem in general, an aspect we leverage in the training within the end-to-end framework. Our approach introduces a neural network architecture that near-optimally solves optimization problems while ensuring feasibility constraints through alternate projections. We prove exponential convergence, approximation guarantees, and generalization bounds for our learning method. This method offers superior computational efficiency, producing high-quality approximations faster and scaling better with problem size compared to existing techniques. Our approach applies to a wide range of optimization problems including deterministic, single-stage as well as two-stage stochastic optimization problems. We illustrate how our proposed method applies to (1) an electricity generation problem using real data from an electricity routing company coordinating the movement of electricity throughout 13 states, (2) a shortest path problem with a computer vision task of predicting edge costs from terrain maps, (3) a two-stage multi-warehouse cross-fulfillment newsvendor problem, as well as a variety of other newsvendor-like problems. † † runningauthor: Cristian et al. † † runningtitle: Meta-optimization for efficient end-to-end learning † † authors: Massachusetts Institute of Technology, [email protected] IBM Research [email protected] Massachusetts Institute of Technology [email protected] IBM Research [email protected] † † affiliation: † † affiliation: † † affiliation: † † affiliation: † † affiliation: † † affiliation: † † affiliation: † † affiliation: 1 Introduction In many operations management problems, several input quantities need to be predicted from historical data prior to determining the corresponding best operational decision. Examples include predicting travel times in a vehicle routing problem, the demand distribution in a supply chain inventory optimization problem, among many others. A popular approach is to estimate these quantities using a machine learning model from historical data along with observed contextual features such as seasonal trends, location and price information among others. This machine learning model is used to create a forecast for a new observation which is subsequently used for decision making. In these problems, the quality of the resultant business objectives, such as the overall costs in the supply chain problem, is often the more important performance indicator, compared to the accuracy of the machine learning models, such as the mean square error of the demand estimate. For example, consider the classical newsvendor problem in which one needs to forecast future demand. There is a holding cost associated with stocking more than the demand, and a backorder cost associated with each unit stocked below demand. Given perfect information about the distribution, the optimal stocking quantity is known to be a particular quantile dependent on the two costs (see Arrow et al. 1951 ). Nevertheless, such closed form solutions are not common in most application settings. For example, once there are multiple products with constraints on the stocking allocation, the optimal solution has no longer a closed form. A predict-then-optimize type of approach may take the following steps. (1) Assume the demand distribution comes from some family of distributions (such as Gaussian) and make a prediction using the mean and variance estimated from the data. Subsequently, one can solve the corresponding stochastic optimization problem. Alternatively (2) one may explicitly learn a distributional forecast (for example, learn the probability of each demand realization) and then subsequently solve the corresponding stochastic optimization problem to determine the allocation quantity. Clearly such distributional forecasts are necessary since using only a point forecast one cannot a priori know the correct statistic of the demand distribution to target. Indeed, simply making point forecasts by minimizing the mean-squared error between the prediction and the observed uncertainty has the potential to produce poor decisions as discussed for example in Cameron et al. 2021 . However, joint prediction and optimization, (also referred to as end-to-end learning), bypasses this issue. The primary goal in this case is to learn a forecast whose corresponding decision has the lowest possible cost (objective function). This goal is exactly incorporated into the training algorithm used to learn the forecasting model. Consider the newsvendor problem as an example. In this problem one needs to be able to implicitly learn the best forecast to make for the “correct quantile” of the distribution as that would minimize cost. At the same time, one needs to keep in mind that any model will always make some error. However, not all errors are the same. For example, suppose the backorder cost is twice that of the holding cost. A model only minimizing mean-squared error between its predictions and the target would identify an over- and under-prediction the same. However, a model trained end-to-end would identify the true cost resulting from a prediction, and that over-prediction results in lower cost. A significant issue in training end-to-end models comes from the required computational complexity. In particular, the loss function is now the output of a decision-making process (to determine the true cost) which is expensive to evaluate. In this paper, we propose a novel approach to end-to-end learning that is more efficient compared to existing methods and achieves similar performance. For many classes of these optimization problems like linear or quadratic, as well as for stochastic optimization problems like the resource allocation problems, where there are over and under utilization penalties, we observe that point forecasts are sufficient when using an end-to-end learning approach. Together with the proposed learning method in this paper, we can now efficiently solve computationally intractable stochastic optimization problems in an end-to-end sense. More specifically, this paper presents the following contributions: 1) Meta-optimization approach to End-to-End for Single and Two Stage Problems: We achieve more efficient end-to-end training by replacing the complex optimization-based loss function with a simpler efficient approximation. We introduce a meta-optimization method to learn better surrogate optimization objectives for which we can compute efficient solutions. We refer to this approach as ProjectNet. We further incorporate this meta-optimization method into an end-to-end learning framework for solving the single-stage stochastic optimization problem. Furthermore, we present a way to extend our approach to two-stage stochastic optimization problems in section 4 . 2) Convergence, Approximation, and Generalization: We prove bounds on how quickly a sequence generated by the ProjectNet architecture converges as well as bounds on the regret of the solution compared to using the exact original optimization problem. In addition, we also prove generalization bounds on how much data is needed to train the ProjectNet model. 3) Point versus Distributional Forecasts for Stochastic Optimization: Moreover, we prove that for a large class of stochastic optimization problems, if one uses a loss-like objective function, then point forecasts through an end-to-end learning approach produce the same optimal solutions