End-to-end Deep Reinforcement Learning for Stochastic Multi-objective Optimization in C-VRPTW Journal: N/A

Abdo Abouelrous, Laurens Bliek, Yaoxin Wu, Yingqian Zhang · 2025

Read End-to-end Deep Reinforcement Learning for Stochastic Multi-objective Optimization in C-VRPTW Journal: N/A by Abdo Abouelrous, Laurens Bliek, Yaoxin Wu, Yingqian Zhang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2512-01518.txt.

Introduction

Abdo Abouelrous and co-authors learn routes for delivery vans under doubt. Their task mixes random travel times with more than one goal. Goals can clash, like cost versus late drops. Real dispatch must weigh those clashes daily.

Their method learns end to end with deep reinforcement learning. It faces travel time doubt head on. The policy adapts routes as conditions shift. It gives dispatchers a front of trade-off choices, not one fixed plan. Learned routing could cut cost and late rates for fleets in busy cities. Multiple goals are kept visible in the output, so a human dispatcher still makes the final call on cost, time, and fairness trade-offs.

Catalog record

CREATORAbdo Abouelrous, Laurens Bliek, Yaoxin Wu, Yingqian Zhang
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE51 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2512-01518

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

End-to-end Deep Reinforcement Learning for Stochastic Multi-objective Optimization in C-VRPTW Journal: N/A Abdo Abouelrous, Laurens Bliek, Yaoxin Wu, Yingqian Zhang Abstract In this work, we consider learning-based applications in routing to solve a Vehicle Routing variant characterized by stochasticity and multiple objectives. Such problems are representative of practical settings where decision-makers have to deal with uncertainty in the operational environment as well as multiple conflicting objectives due to different stakeholders. We specifically consider travel time uncertainty. We also consider two objectives, total travel time and route makespan, that jointly target operational efficiency and labor regulations on shift length, although more/different objectives could be incorporated. Learning-based methods offer earnest computational advantages as they can repeatedly solve problems with limited interference from the decision-maker. We specifically focus on end-to-end deep learning models that leverage the attention mechanism and multiple solution trajectories. These models have seen several successful applications in routing problems. However, since travel times are not a direct input to these models due to the large dimensions of the travel time matrix, accounting for uncertainty is a challenge, especially in the presence of multiple objectives. In turn, we propose a model that simultaneously addresses stochasticity and multi-objectivity and provide a refined training mechanism for this model through scenario clustering to reduce training time. Our results show that our model is capable of constructing a Pareto Front of good quality within acceptable run times compared to three baselines. We also provide two ablation studies to assess our model’s suitability in different settings. 1 Introduction Real-life Combinatorial Optimization (CO) problems, such as routing problems, are often characterized by certain challenging features. Examples of such features include stochasticity and the presence of several (conflicting) objectives simultaneously. In applications like the Vehicle Routing Problem (VRP), accounting for these features during decision-making is important. Otherwise, sub-optimal decisions may lead to elevated costs. Furthermore, computational budgets are often limited in practice as solutions need to be generated quickly in accordance with operational requirements ( 11 ) . To that end, one seeks optimization techniques that are not only capable of delivering solutions efficiently but also delivering near-optimal solutions in the presence of the aforementioned features. In the literature, numerous techniques have been proposed to solve CO problems, ranging from hand-crafted heuristics to Machine Learning (ML) models that are independently able to generate solutions - also known as end-to-end methods. The spectrum also covers hybrid methods, which incorporate both heuristics and ML models. ML models are particularly useful in cases where problem parameters follow a known distribution. In such cases, an ML model can be trained on a dataset of problems from this distribution and used to generate solutions for problem instances arising in the future. The ability of ML models to generate solutions statistically - even when the solution distribution is unknown beforehand - by leveraging data from the operational environment to make decisions in a principled fashion, has been repeatedly stressed in the literature ( 6 ; 10 ; 20 ) . This is because ML models have the capacity to learn from the collective expert knowledge. Through learning common solution structures and relationships to problem parameter input, ML model can continuously solve different problems. Although the training resources required are not trivial, ML models can be quickly applied for solving thereafter ( 38 ) . This is in stark contrast to hand-crafted heuristics which, generally, do not leverage information from the solution structure ( 8 ) and may be challenging to repeatedly develop and/or replicate for newer instances 2 . For the large part, the CO problems treated in the literature by ML models were deterministic and had a single objective. Real-life problems, contrarily, are often characterized by uncertainty and multiple (conflicting) objectives. The objective of our study is to extend the framework of ML in optimization to settings that jointly combine the stochastic and multi-objective features. In our estimation, this is an under-studied application with significant industrial relevance. Solving a multi-objective problem requires the definition of a Pareto Front ( 21 ) , comprising of a set of Pareto optimal solutions by which improving one objective can not be done without worsening other objectives. Solutions would then have to be validated on a sample of scenario realizations to estimate their quality and feasibility. In an optimization context, this poses a challenge that requires novel intervention. This is because multi-objectivity induces the generation of several solutions to estimate the Pareto Front, while stochasticity imposes that each solution be feasible with a good score for a large number of stochastic realizations. The repeated evaluation of multiple solutions, in this setting, is a cumbersome task that requires careful choices in the optimization methodology to maintain computational costs within practical limits. In response, we present a framework that addresses the given computational issues in this paper to solve Capacitated Vehcile Routing Problem with Time Windows (C-VRPTW). We specifically focus on stochasticity in travel times and consider two objectives that are travel time-based, although our method could account for more different objectives. The major contributions prescribed by this paper are: • Presenting the first end-to-end (independently constructs solution without interference of a heuristic) deep-learning model that jointly treats stochasticity and multiple objectives for routing. • Providing a retraining mechanism for parameter uncertainty through an active search that considers multiple objectives rather than just one objective. • Presenting a scenario-clustering technique that enhances the computational efficiency of model retraining during active search. The remainder of the paper is organized as follows. Section 2 introduces previous research. Section 3 describes the relevant problem in detail. Section 4 outlines our methodology. Section 5 presents numerical experiments and results. Lastly, Section 6 asserts the conclusions. 2 Previous Work We restrict our attention to ML applications in stochastic and/or multi-objective cases for routing. These models generally rely on concepts of Reinforcement Learning (RL) to make decisions. These approaches fall into two major categories, hybrid and end-to-end. Hybrid models integrate a ML model with a heuristic to enhance the heuristic’s performance. The way in which the ML model is applied largely depends on the heuristic at hand. A common example is that the RL agent selects a heuristic with some probability from a set of discrete heuristics. In contrast, End-to-end models are able to independently solve a routing problem without interference of a heuristic. When deciding on the ordering of node visits, the decision to visit a node j after node i is based on a probability p_{ij} . Such a probability may be determined by features of nodes i and j , the status of the current (partial) solution and the estimated corresponding reward. In both categories, the decisions are determined by interaction of the problem input with the ML model’s parameters which are optimized during model training to maximize the final reward. For the multi-objective setting, works like 34 and 9 explore applications to routing with hybrid methods through genetic algorithms. 35 , on the other hand, focus on hyper-heuristic selection by the RL agent to complete a solution. In the Stochastic setting, 5 make use of simulation-augmented optimization for urban-routing. 15 employ a hybrid approach with a genetic algorithm to address a dynamic problem with stochastic customers. Works integrating both stochastic and

Read the complete text