MORSE: Multi-Objective Reinforcement Learning via Strategy Evolution for Supply Chain Optimization

Niki Kotecha, Ehecatl Antonio del Rio Chanona · 2025

Read MORSE: Multi-Objective Reinforcement Learning via Strategy Evolution for Supply Chain Optimization by Niki Kotecha, Ehecatl Antonio del Rio Chanona (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-06490.txt.

Introduction

Balancing cost, service, and green goals in supply chains is the challenge addressed by Niki Kotecha and one co-author in a 2025 operations paper. Classic optimization tools react slowly when conditions change without warning. Managers need policies that can switch as goals shift during the day.

Their method, MORSE, blends reinforcement learning, where software learns by trial and feedback, with evolutionary search over the settings of policy networks. The search produces a front of policies, each trading the goals off differently, so managers can switch among them in real time. A risk measure called conditional value-at-risk makes choices cautious under uncertainty. In case studies, including an inventory problem, the approach adapts to changing conditions and beats leading methods, offering a sturdier way to run supply chains.

Catalog record

CREATORNiki Kotecha, Ehecatl Antonio del Rio Chanona
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE71 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2509-06490

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

MORSE: Multi-Objective Reinforcement Learning via Strategy Evolution for Supply Chain Optimization Niki Kotecha, Ehecatl Antonio del Rio Chanona Abstract In supply chain management, decision-making often involves balancing multiple conflicting objectives, such as cost reduction, service level improvement, and environmental sustainability. Traditional multi-objective optimization methods, such as linear programming and evolutionary algorithms, struggle to adapt in real-time to the dynamic nature of supply chains. In this paper, we propose an approach that combines Reinforcement Learning (RL) and Multi-Objective Evolutionary Algorithms (MOEAs) to address these challenges for dynamic multi-objective optimization under uncertainty. Our method leverages MOEAs to search the parameter space of policy neural networks, generating a Pareto front of policies. This provides decision-makers with a diverse population of policies that can be dynamically switched based on the current system objectives, ensuring flexibility and adaptability in real-time decision-making. We also introduce Conditional Value-at-Risk (CVaR) to incorporate risk-sensitive decision-making, enhancing resilience in uncertain environments. We demonstrate the effectiveness of our approach through case studies, showcasing its ability to respond to supply chain dynamics and outperforming state-of-the-art methods in an inventory management case study. The proposed strategy not only improves decision-making efficiency but also offers a more robust framework for managing uncertainty and optimizing performance in supply chains. K eywords Multi Objective Reinforcement Learning \cdot Evolutionary Algorithms \cdot Inventory Control \cdot Supply Chain Optimization 1 Introduction In recent years, the field of Process Systems Engineering (PSE) has experienced a significant shift towards advanced optimization techniques that are capable of handling the inherent complexities of modern systems, whether in process control ( Liñán and Ricardez-Sandoval 2025 ) , sustainable supply chains ( Qiu et al. 2024 ; Grossmann and Guillén-Gosálbez 2010 ; Rangel-Martinez et al. 2021 ) , or other industrial operations ( Szatmári et al. 2024 ) . A key challenge in PSE is the need to optimize multiple, often conflicting objectives simultaneously, such as minimizing costs, maximizing throughput, ensuring quality, and reducing environmental impact. This complexity has become particularly evident as the development of sustainable practices has emerged as a key strategic focus for many organizations, driven by the increase in regulatory requirements and pressure from consumers to adopt environmentally friendly practices. As industries strive towards making their value chains more sustainable, the integration of these principles into every aspect of their operations, particularly in decision-making, becomes essential. The historical roots of multi-objective optimization trace back to the work of Vilfredo Pareto in the late 19th century ( Pareto 1935 ) . Pareto efficiency, or Pareto optimality, refers to a situation where no objective can be improved without worsening another. In the context of multi-objective optimization, Pareto-optimal solutions represent a set of decisions where the trade-offs between competing objectives are balanced in the most efficient way. This foundational concept forms the basis of the modern approach to multi-objective optimization, where the goal is to generate a set of Pareto-optimal solutions for decision-makers to choose from, based on their specific priorities ( Gunantara 2018 ) . While traditional MOO method, including Linear Programming (LP) ( Özceylan and Paksoy 2013 ) and Evolutionary Algorithms (EAs) ( Liao et al. 2011 ) , are well-established, the increasing complexity of real-world process systems requires dynamic and adaptable solutions. This has led to the emergence of Dynamic Multi-Objective Optimization (DMOO), which seeks to optimize systems that evolve over time in response to changing environmental, operational, and economic conditions. DMOO methods adapt to evolving conditions, re-optimizing solutions as objectives shift, which is particularly important for process systems engineering where decisions must be continuously adapted to accommodate variations in process dynamics, disturbances, and uncertain operating conditions which require fast and flexible decision-making. In such cases, the Pareto set must be recomputed as the environment changes. However, in fast-paced environments, the computational overhead involved in repeatedly solving DMOO problems can hinder fast, adaptive, real-time decision-making. This motivates the use of multi-objective reinforcement learning (MORL), which builds on RL principles to address multi-objective decision-making. Recent advances in MORL have demonstrated its ability to solve multi-objective decision-making problems in a series of applications in process systems engineering domains ( Li et al. 2022 ) , inventory management ( Qiu et al. 2024 ) , robotics control ( Xu et al. 2020 ) and energy management ( Wu et al. 2023 ) . 1.1 Related Work Traditional Multi-Objective Optimization (MOO) approaches focus on finding a set of optimal solutions, known as the Pareto optimal, that balance trade-offs between different objectives such as cost, service level and environmental emissions. The collection of all Pareto optimal solutions forms the Pareto Front, which provides decision-makers with a range of non-dominated solutions, allowing them to choose the most appropriate solution based on their system requirements and preference. Methods such as Linear Programming (LP) ( Özceylan and Paksoy 2013 ) , Goal Programming ( Dutta and Kumar 2015 ) , and Evolutionary Algorithms (EAs) ( Liao et al. 2011 ) , including Non-dominated Sorting Genetic Algorithm II (NSGA-II) ( Hnaien et al. 2010 ) , have been widely used to approximate the Pareto front effectively. These approaches focus on two key criteria: (1) proximity - ensuring solutions closely approximate the true Pareto front, and (2) diversity - ensuring solutions span a wide range of trade-offs across the objective space. MOO has been widely applied in inventory management, integrating numerous objectives such as cost efficiency, service levels, and sustainable practices ( Aslam and Amos 2010 ) . For example, Agrawal et al. 2020 explored a deterministic multi-objective optimization process that targeted multi-item inventory management, emphasizing the balance between reducing shortages and controlling costs through multi-criteria decision-making techniques. Recent literature has also explored the integration of sustainability principles into inventory management paradigms, addressing energy consumption and risk of shortages across multi-storage supply chains ( Xie et al. 2024 ; Jayarathna et al. 2021 ; Srivastav and Agrawal 2016 ) . This highlights the trend towards incorporating sustainability metrics, emphasizing the need of addressing environmental issues alongside traditional inventory management strategies. Simulation-based optimization approaches have also emerged to develop frameworks that better equip decision-makers by evaluating various inventory strategies whilst accounting for multiple conflicting objectives ( Tsai and Chen 2017 ; Amodeo et al. 2009 ; Lee et al. 2005 ) . However, as the number of objectives increases, multi-objective optimization becomes computationally challenging due to the curse of dimensionality, requiring more sophisticated methods such as decomposition-based algorithms ( Abbassi et al. 2022 ) and indicator-based approaches ( Yang et al. 2016 ) . In dynamic environments, where objectives change over time, MOO techniques can be extended to dynamic multi-objective optimization (DMOO) methods. DMOO methods adapt to evolving conditions, re-optimizing solutions as objectives shift, which is particularly important in the context of supply chains, where external disruptions and market fluctuations require fast and flexible decision-making. In such cases, the Pareto set must be recomputed as the environment changes. However, in fast-paced supply chains, the computational overhead involved in repeatedly solving DMOO problems can hinder fast, adaptive, real-time decision-making ( Helbig et al. 2016 ) . 1.1.1 Multi-Objective Reinforcement Learning (MORL) To address these shortcomings, multi-objective reinforcement learning (MORL) has emerged as a promising alternative that integrates the principles of DMOO with RL ( Hayes et al. 2022 ) . Recent advances in

Read the complete text