An improved Q-learning approach for rescue path planning in mass casualty incidents under damaged road network conditions Authors: Sheng Wang, Jingrong Yang, Peng Yang Published: 2026 ABSTRACT AbstractEmergency rescue path planning faces severe challenges due to the uncertainty of the road network and the complexity of road damage following mass casualty incidents (MCIs) such as earthquakes. Traditional static path planning algorithms struggle to cope with such dynamic environments, while standard reinforcement learning methods often cause agents to fall into local optima, failing to meet rescue safety requirements. To address this dilemma, this paper proposes an improved Q-learning algorithm that integrates Adaptive Simulated Annealing (ASA) and Experience Replay (ER) mechanisms. The algorithm dynamically regulates the exploration degree via the ASA strategy, ensuring the agent can escape local traps in complex damaged environments. Furthermore, it incorporates the ER mechanism to reuse key path experiences, thereby improving convergence speed. Experimental results demonstrate that the proposed method demonstrates efficient online re-planning capabilities in response to structural environmental updates. It provides high-confidence decision support for time-critical post-disaster rescue, balancing the trade-off between accessibility and safety.Subject terms: Engineering, Natural hazards Introduction Mass casualty incidents (MCIs), typically triggered by natural disasters, technological accidents, or social conflicts, are characterized by their sudden onset and the difficulty of early warning. In recent years, the increasing frequency of natural disasters and man-made accidents worldwide has made MCIs increasingly prevalent1. According to data from emergency event databases, in 2024 alone, the world experienced 393 natural disasters, resulting in over 16,000 deaths and affecting as many as 167 million people2. In these incidents, the “golden hour” for rescue is extremely precious. The ability of emergency response units to reach the scene promptly is a critical determinant of victim survival rates and the overall success of the rescue operation. Therefore, establishing a rapid and safe lifeline for rescue forces remains one of the most critical challenges in MCI emergency response. MCIs, particularly geological disasters like earthquakes, often devastate urban infrastructure, most notably by crippling the road network. This destruction manifests not only as physical disruptions—such as collapse and damage—but, more critically, as extreme environmental uncertainty. Pre-disaster maps and navigation data are rendered instantly obsolete. The actual navigability of roads becomes completely unknown, and the situation remains highly dynamic, subject to continual change from aftershocks and secondary disasters. This redefines the core emergency response challenge, shifting it from a simple shortest path problem to a complex dynamic decision-making task within an unknown, high-risk environment characterized by incomplete information3. Consequently, rescue vehicles must be capable of dynamically exploring and planning optimal paths that simultaneously balance safety and efficiency, all while navigating around hazardous areas in real-time. While conventional path planning algorithms, such as A* or Dijkstra, excel in static and fully observable environments, they rely heavily on precise a priori map information. In the event of unforeseen environmental changes, these algorithms are often rendered ineffective by erroneous information4. Consequently, the research community has shifted towards model-free Reinforcement Learning (RL) methods. RL was initially applied extensively in disciplines such as game theory, information theory, control theory, and operations research5. Over time, its application expanded to autonomous mobile robotics and path planning6. Q-learning, as a representative algorithm, is theoretically capable of autonomously learning optimal paths in unknown environments through its trial-and-error interaction mechanism7. However, when standard Q-learning is directly applied to high-risk, complex real-world scenarios like MCIs, its inherent deficiencies are exposed. Firstly, its inefficient exploration-exploitation mechanism is highly prone to converging on local optima. In a rescue context, this translates to a catastrophic risk: leading teams into dead ends or hazardous zones, potentially causing catastrophic consequences. Secondly, its slow learning process makes it incapable of quickly generating a reliable route plan within the brief decision-making window available for emergency response. To address the aforementioned research gap and the pressing need for rapid, reliable path decisions in uncertain road networks, this paper proposes a novel emergency decision-support method based on an improved Q-learning algorithm. The core advancement of this approach is the synergistic integration of Adaptive Simulated Annealing (ASA) and Experience Replay (ER) mechanisms, which specifically overcomes the sparse reward and local optima traps inherent in damaged road environments. While conventional Q-learning often struggles with state-space bottlenecks when key road segments are disconnected, the ASA strategy introduces a dynamic exploration mechanism that facilitates escape from dead-end states. Concurrently, the ER mechanism ensures that rare but critical high-safety rescue trajectories are preserved and repeatedly utilized to stabilize policy convergence. The primary contributions of this work are summarized across three dimensions: Methodological innovation: A synergistic reinforcement learning framework is proposed, which uniquely couples a two-stage adaptive cooling strategy with an ER mechanism. This design effectively addresses the dual challenges of high-risk local optima traps and extreme reward sparsity in fragmented post-disaster topologies. Methodological innovation: A synergistic reinforcement learning framework is proposed, which uniquely couples a two-stage adaptive cooling strategy with an ER mechanism. This design effectively addresses the dual challenges of high-risk local optima traps and extreme reward sparsity in fragmented post-disaster topologies. Technological integration: This study systematically integrates the autonomous learning capability of Q-learning, the probabilistic “jump” mechanism of Simulated Annealing, and the sample-reuse efficiency of ER. This integration transforms independent baseline techniques into a cohesive, resilient online re-planning architecture. Technological integration: This study systematically integrates the autonomous learning capability of Q-learning, the probabilistic “jump” mechanism of Simulated Annealing, and the sample-reuse efficiency of ER. This integration transforms independent baseline techniques into a cohesive, resilient online re-planning architecture. Scenario-tailored design: A quantitative environment model for MCI rescue is established by introducing the Road Connectivity Reliability (RCR) metric and a quadratic risk penalty. This modeling choice facilitates the mapping of non-linear physical road destruction into a weighted state space, ensuring safety-critical path planning. Scenario-tailored design: A quantitative environment model for MCI rescue is established by introducing the Road Connectivity Reliability (RCR) metric and a quadratic risk penalty. This modeling choice facilitates the mapping of non-linear physical road destruction into a weighted state space, ensuring safety-critical path planning. The remainder of this paper is organized as follows: The section "Related work" provides a review of path planning algorithms for MCI emergency response, with a particular focus on the current status of reinforcement learning applications in dynamic environments. The section "Problem formulation and solution method" elaborates on the theoretical framework of the traditional Q-learning algorithm and details the proposed improvements: the data augmentation mechanism based on Experience Replay and the dynamic exploration strategy based on Adaptive Simulated Annealing. The section "Experiments and results" validates, through a series of comparative simulations and ablation studies, that the improved algorithm can efficiently accomplish path planning tasks in the specific context of MCIs. Finally, the section "Conclusion" summarizes the research work and discusses directions for future research. Related work The uncertainties inherent in MCI emergency response pose a severe challenge to conventional path planning algorithms, prompting a research shift towards model-free Reinforcement Learning methods. Q-learning, in particular, has garnered significant attention due to its autonomous learning capabilities. However, standard Q-learning exhibits limitations when directly applied to MCI scenarios. Consequently, the current research focus has converged on improving the algorithm to enhance its effectiveness in this specific context. Routing challenges in emergency response In MCIs, emergency vehicle routing is a critical determinant of successful rescue operations8 and represents a core issue within the fields of emergency logistics and disaster management. Traditionally, research in this domain has centered on optimization methods derived from Operations Research (OR). This typically involves modeling the emergency routing challenge as a variant of the Vehicle Routing Problem (VRP) and employing graph-based or heuristic algorithms to derive solutions. However, the post-disaster environment diverges significantly from conventional routing scenarios that typically prioritize the shortest path; its most salient feature is extreme uncertainty. As Wang noted9, factors such as demand volume, demand locations, and vehicle travel times become highly unpredictable due to disaster impacts, while the scarcity of real-time data renders precise planning exceptionally difficult. Furthermore, severe traffic congestion emerges as a major impediment, occurring even where roads remain physically intact. This congestion is often driven by massive evacuation flows, the influx of rescue vehicles, and the reduced capacity of damaged road segments. Bahar et al.10 identified congestion as a critical factor impeding the efficiency of victim transport during earthquake response. By employing the Bureau of Public Roads (BPR) function to quantify the impact of congestion on travel times, their work demonstrated that the geographically shortest path is frequently not the temporally optimal one. The physical destruction and extreme informational uncertainty caused by disasters render a priori map-based methods ineffective11. While heuristics like Genetic Algorithms (GAs) have been applied to post-disaster VRP, they often suffer from unstable and suboptimal solutions. Thus, robust path planning in incomplete and highly uncertain networks remains a significant technological bottleneck in emergency management. This challenge has driven the research community to pivot towards model-free machine learning, where RL has shown immense potential. Q-learning, as a representative algorithm, is a foundational approach valued for its ability to learn optimal policies through direct, trial-and-error interaction with unknown environments, thereby autonomously planning paths to the target12. For instance, Rocha et al.13 proposed a method to dynamically adjust Q-learning iterations for online planning. Their approach aims to adaptively determine sufficient iterations based on environmental complexity, seeking a balance between solution quality and reduced decision time in unknown complex environments. Similarly, focusing on Unmanned Aerial Vehicle (UAV) search and rescue in unknown environments, Wu et al.14 designed an adaptive-velocity Q-learning algorithm. This method uses composite rewards to adaptively adjust flight speed and incorporates sub-domain search and Q-table initialization strategies to accelerate target discovery and convergence to the optimal path. The SARSA algorithm15 is another representative RL approach. Unlike Q-learning, SARSA eliminates the maximization operator in its update rule, tending to plan trajectories that, while potentially longer, maintain an optimal safety distance from hazardous areas. For example, Mohan et al.16 proposed an Iterative SARSA algorithm for mobile robot navigation in unknown environments. By alternately executing multiple SARSA iterations between the start and goal points to progressively refine the path, this method aims to address the execution risks of standard Q-learning and the sub-optimality of standard SARSA, ultimately achieving an optimal path that ensures safety. While the aforementioned studies demonstrate the potential of Q-learning and its variants in handling dynamics, improving online efficiency, and accelerating convergence, directly applying these methods to the specific MCI scenarios addressed in this paper remains challenging. The MCI environment is not merely unknown or dynamic; more critically, it is characterized by high risk-uncertainty stemming from road damage and extremely stringent temporal constraints. Enhanced exploration strategies In MCI rescue, merely finding a suboptimal path is wholly insufficient; discovering the global optimum is imperative to avoid critical delays. A core bottleneck restricting standard Q-learning in routing applications is the efficacy of its exploration mechanism, notably the widely adopted ε-greedy strategy. This strategy introduces a probability, ε, for selecting random actions, aiming to escape local policies and explore the broader state space. However, this simplistic mechanism presents a fundamental dilemma: on one hand, a low ε value leads to insufficient exploration, making the algorithm highly prone to premature convergence into a local optimum trap rather than the global optimum17. Conversely, a high or fixed ε value compromises convergence stability. The algorithm continues to perform unnecessary random exploration even in the late stages of training, leading to suboptimal pathing efficiency and a failure to stabilize. In the high-risk environment of MCI rescue characterized by a complex state space, converging on a local optimum is tantamount to selecting a suboptimal or even hazardous route, potentially resulting in catastrophic outcomes. To address the aforementioned problems, researchers have proposed various optimization schemes for exploration mechanisms. The most straightforward improvement is to implement a dynamic rather than static exploration factor, ε, which typically maintains a high value during the initial learning phase and gradually decays as training episodes increase18. For instance, Li et al.19 segmented the learning process into three distinct phases—exploration, exploration-exploitation, and exploitation. By dynamically adjusting the exploration factor across these stages, this staged mechanism enables the agent to adopt exploration strategies that are better suited to specific learning periods, thereby effectively balancing the algorithm’s strategic requirements. Furthermore, researchers have designed more sophisticated dynamic adjustment mechanisms. Notably, Ben-Akka et al.20 proposed a novel ε-greedy decay strategy that dynamically adjusts the exploration rate based on cumulative rewards, aiming to ensure a smooth transition from exploration to exploitation and to stabilize the agent’s behavior. A second class of methods selects actions probabilistically based on Q-value estimates, with Boltzmann exploration being a representative example. To combine the advantages of ε-greedy and Boltzmann exploration, Fallooh et al.21 proposed a hybrid strategy that blends both methods during the exploration phase to achieve a better balance between exploration and exploitation. However, Boltzmann strategies suffer from significant drawbacks: they are highly sensitive to the relative magnitude of Q-values, requiring careful tuning of the temperature parameter, and may degenerate into a purely greedy policy when Q-value disparities are significant. More advanced strategies attempt to actively guide exploration towards regions of the state space with the highest epistemic uncertainty or potential value. For instance, the Upper Confidence Bound (UCB) algorithm22 estimates a confidence interval for each action and preferentially selects the one with the highest upper bound. Building on this, Sheng et al.23 proposed an improved ε-UCB action selection policy that employs a neural network to model the uncertainty of state-action pairs, thereby effectively enhancing exploration efficiency. Similarly, methods such as count-based exploration24 or intrinsic curiosity25 incentivize agents to explore novel or unpredictable states by introducing additional reward signals. Zhou et al.25, for example, integrated an Intrinsic Curiosity Module (ICM) into DQN, utilizing prediction errors to quantify state novelty and guide UAVs in performing more effective coverage path planning. While these methods theoretically enable more effective exploration, they are often hindered by high computational complexity or the requirement for complex models to quantify uncertainty. A distinct and sophisticated approach involves integrating the principles of Simulated Annealing (SA) into the decision-making process. For instance, Mao et al.26 incorporated SA principles into the action selection policy of Q-learning. In their method, the agent probabilistically accepts a suboptimal random action based on a temperature parameter that changes dynamically, thereby increasing environmental exploration capabilities and better balancing the exploration-exploitation dilemma. Zhong and Wang27 utilized SA principles to guide the adjustment of a dynamic exploration factor. Furthermore, Shi et al.28 combined an improved SA algorithm with Ant Colony Optimization (ACO) to solve the coverage path planning problem for cleaning robots, demonstrating SA’s potential in complex spatial searches. Recently, Zhong and Wang27 proposed an improved Q-learning algorithm, termed Heuristic and Simulated Annealing-based Q-learning (HSA-QL), which incorporates a heuristic reward based on Euclidean distance and a simulated annealing-guided exploration factor to enhance convergence in cross-regional planning. Compared to the random exploration of ε-greedy, this approach is inherently more goal-directed and possesses a greater potential to escape deep local optimum traps. While these improved strategies enhance the exploration performance of Q-learning in various aspects, applying them to MCI response scenarios requires cautious evaluation. The MCI environment is not only complex and filled with traps formed by high-risk road segments, but also imposes stringent and often conflicting requirements for both real-time decision-making and global optimality. Enhanced learning efficiency Emergency decision-making imposes exceptionally stringent requirements on timeliness. However, the standard Q-learning algorithm is hindered by its online mechanism where experiences are utilized only once and then immediately discarded. This results in low sample efficiency and a sluggish learning process, making it difficult to rapidly generate reliable solutions within the brief decision-making window of an MCI rescue. To address the core challenges of slow convergence and low learning efficiency, the research community has explored solutions from multiple dimensions. One of the most direct methods to enhance learning efficiency is Reward Shaping—optimizing the reward function to transition from sparse to dense, thereby providing the agent with clearer gradient guidance. For instance, Jaramillo-Martínez et al.29 focused on reward function strategies, proposing a function that incentivizes agents to select paths with fewer turns, thereby reducing total distance while simultaneously accelerating learning. Hao et al.30 combined Q-learning with the Artificial Potential Field (APF) method and designed a dynamic reward function specifically to improve convergence speed. In the multi-agent context, Song et al.31 proposed a cooperative reward shaping technique that designs rewards by evaluating an agent’s influence on its neighbors, fostering cooperation and enhancing overall learning efficiency. Learning efficiency is governed not only by reward signals but also by the stability of the value function update mechanism. Standard Q-learning and its Deep Q-Network (DQN) variants suffer from a known overestimation bias, wherein Q-values tend to be systematically overestimated. Van Hasselt et al.32 explicitly identified this issue and demonstrated that such overestimation is detrimental to performance. To address this, Double Q-learning and its deep variant, DDQN, were introduced. These methods effectively mitigate Q-value overestimation by decoupling action selection from value evaluation, thereby improving algorithmic stability and convergence efficiency. This approach has found application in emergency scenarios; for example, Zhou et al.33 employed a Double DQN (DDQN) framework to guide multi-agent path planning in a virtual fire emergency simulation. Despite the stability improvements offered by the aforementioned methods, they fail to address the fundamental limitation of data inefficiency in standard Q-learning, where experiences are typically discarded immediately after a single update. Experience Replay (ER) serves as a cornerstone for enhancing sample efficiency, not only in Deep Reinforcement Learning but also in tabular settings. By constructing a replay buffer to store historical data, ER allows the agent to revisit past transitions and perform offline updates without direct interaction with the real environment. This mechanism maximizes the utility of individual interactions, significantly accelerating policy propagation within complex state spaces. A representative application is found in the work of Jiang et al.34, who integrated ER with Deep Q-Networks (DQN) and heuristic knowledge for robotic path planning. Their results demonstrated that the inclusion of ER enabled the robot to converge to the optimal policy in significantly less time and fewer steps. Building on this foundation, researchers have developed a more advanced variant known as Prioritized Experience Replay (PER), which prioritizes the learning of experiences with higher Temporal Difference (TD) errors. Studies by Gök35 and Zhu36 both combined PER with Dueling Double Deep Q-Networks (D3QN) for dynamic path planning, further corroborating the significant advantages of PER in accelerating convergence and enhancing path quality. Problem formulation and solution method To effectively address the challenges of emergency rescue path planning in uncertain road networks, this study proposes a novel algorithm built upon standard Q-learning that integrates an improved learning mechanism with an enhanced exploration strategy. The overall framework of the proposed algorithm is illustrated in Fig. 1. Specifically, it introduces an Experience Replay (ER) mechanism to optimize the learning process, thereby addressing the challenge of decision efficiency. Concurrently, it integrates an Adaptive Simulated Annealing (ASA) strategy to optimize the exploration process, addressing the challenge of decision quality. This section first formulates the MCI rescue problem as a Markov Decision Process (MDP) and subsequently elaborates on the individual components of the proposed algorithm. The overall framework of the proposed algorithm, integrating ASA for exploration and ER for learning. Environment modelling Grid map modeling Environment modeling is the foundational process of translating the physical space of an MCI rescue into a mathematical construct that a Reinforcement Learning (RL) agent can comprehend and interact with. Various modeling approaches exist in the field of path planning, such as geometric feature-based, topological, and grid-based methods. Geometric methods often incur high computational costs in complex or dynamic environments, while topological methods disregard geometric information, making them ill-suited for MCI rescue scenarios that require precise routing37. This study adopts the grid-based method as the foundational modeling tool. By discretizing the complex continuous environment into a series of uniform grid cells, this approach aligns perfectly with the discrete state space definition of RL, where each cell constitutes an independent state38. Crucially, the grid-based method possesses robust capabilities for environmental information representation, allowing rich attributes to be assigned to each state. This feature is vital for MCI scenarios as it enables the model to transcend traditional binary representations. Instead, it assigns a reliability attribute to each grid cell, thereby quantifying the degree of damage. The specific map used in this study is illustrated in Fig. 2. Visualization of the grid-based environment, where black cells represent impassable obstacles. Quantifying road reliability The core challenge in MCI scenarios is the environmental uncertainty wrought by the disaster. To quantify this uncertainty within the grid model, this study introduces the key metric of RCR. This metric is used to characterize the risk an agent incurs when traversing a given grid state. Using seismic disasters as a case example, we ground our model in the official Classification of Seismic Damage for Lifeline Engineering standard. This standard categorizes road damage into five distinct levels: Destruction, Severe Damage, Moderate Damage, Slight Damage, and Basically Intact. These five levels are then mapped to their corresponding reliability values as specified by the Chinese Seismic Intensity Scale. The resulting RCR values used in this study are presented in Table 1. Table 1Road Connectivity Reliability Values Based on Damage Levels.LevelDamage CategoryDamage DescriptionConnectivity Reliability1Basically IntactRoad surface undamaged or with minimal damage.1.02Slight DamageRoad surface shows slight upheaval and/or subsidence.0.83Moderate DamageRoad surface exhibits noticeable upheaval.0.64Severe DamageSignificant road subsidence, large cracks, and obvious upheaval. Traffic is impeded.0.25DestructionRoad has major dislocations or ruptures; subsidence or suspension exceeds 50 cm. Reconstruction is required.0 Road Connectivity Reliability Values Based on Damage Levels. For practical implementation in the model, this study further partitions the aforementioned reliability values into distinct ranges, as presented in Table 2. These ranges are used to define the Damage Degree of each grid cell. Table 2Damage Degree and Corresponding Ranges.Damage Degree Basically Intact Slight Damage Moderate Damage Severe Damage Destruction Damage Degree and Corresponding Ranges. Specifically, the parameter denotes the connectivity reliability index of a given road grid cell. While the five damage categories are strictly derived from the official Classification of Seismic Damage for Lifeline Engineering standard, the specific numerical ranges for presented in Table 2 are empirically calibrated to model the non-linear impact of road damage on rescue operations. In post-disaster scenarios, the degradation of traffic capacity is not linearly proportional to the visual damage level. Minor structural damage typically causes marginal delays, justifying the narrow upper-bound intervals. Conversely, moderate to severe structural failures induce exponential increases in traversal risk and temporal uncertainty. Consequently, these high-risk states are distributed across a broader lower-bound spectrum. Furthermore, when integrated with the quadratic penalty function (Eq. 2), this specific parameterization mathematically amplifies the traversal costs of severely degraded zones, thereby compelling the reinforcement learning agent to synthesize a strictly risk-averse routing policy. By establishing these definitions, this study successfully translates the problem of physical uncertainty in MCI scenarios into a quantifiable mathematical model on the grid map. The resulting MCI-affected map is illustrated in Fig. 3. Visualization of the quantified MCI environment. MDP formulation of the MCI rescue problem To employ RL in finding the optimal rescue path, the MCI emergency response environment must first be formally formulated as an MDP. An MDP is formally constituted by a quadruple (S, A, P, R), which represents the four core components: the State Space (S), the Action Space (A), the State Transition function (P), and the Reward Function (R). In the context of our MCI routing problem, these components are defined as follows: State Space(S): Represents the agent’s (rescue vehicle’s) location on the grid map, defined by its positional coordinates . Action Space(A): Represents the discrete set of actions the agent can execute,. (These actions typically correspond to moving Up, Down, Left, and Right on the grid map). State Transition(P): This is the state transition function, = ,which defines the probability of transitioning from state to state after taking action. Reward Function (R): The reward function is a critical component of this research, as it directly guides the agent’s learning objective. It must align with the practical demands of MCI rescue, which require a consistent balance between efficiency and safety. The total reward function R is therefore formulated as a composite of two components: a path guidance reward, , and a risk-avoidance penalty, Path guidance reward : This reward is determined by the agent’s position relative to the MCI site after executing action . In this context, the MCI site represents the target destination for the rescue mission. Approaching this target yields a reward, while moving away results in a penalty. Here, and represent the Euclidean distances calculated between the agent’s spatial coordinates and the MCI site at the current time step t and the subsequent time step t+1, respectively. Path guidance reward : This reward is determined by the agent’s position relative to the MCI site after executing action . In this context, the MCI site represents the target destination for the rescue mission. Approaching this target yields a reward, while moving away results in a penalty. Here, and represent the Euclidean distances calculated between the agent’s spatial coordinates and the MCI site at the current time step t and the subsequent time step t+1, respectively. This design provides effective gradient guidance for the agent within the unknown environment, preventing the blind exploration often caused by sparse rewards39. 2)Risk-avoidance penalty : A non-linear penalty is assigned based on the road connectivity reliability, Risk-avoidance penalty : A non-linear penalty is assigned based on the road connectivity reliability, In practical implementation, the specific value required to calculate the risk-avoidance penalty at each step is dynamically obtained by querying the pre-quantified grid map based on the agent’s real-time spatial coordinates. The purpose of this design is twofold: (1) when , no penalty is applied. (2) When , the penalty value increases quadratically as the degree of damage increases. The coefficient ensures that the penalty for high-risk segments is sufficiently salient, effectively deterring the agent from traversing severely degraded areas. In this formulation, the quadratic penalty applied to is specifically designed to capture the highly non-linear escalation of risk in post-disaster environments. Physically, a transition from ‘Slight Damage’ to ‘Severe Damage’ does not merely scale travel time proportionally; it exponentially increases the likelihood of critical failures such as vehicle breakdowns, secondary accidents, and unpredictable delays. By squaring the reliability deficit , the reward function mathematically acts as a strict risk-averse filter. It penalizes highly uncertain segments so heavily that the agent is naturally driven to prioritize geographically longer, but structurally intact and safer, detours. Additionally, to provide the agent with a clear signal regarding task success or failure, terminal rewards are established: When the agent successfully reaches the MCI site, it receives a large-magnitude positive reward, Conversely, if the agent fails the task by colliding with an obstacle or the map boundary, it receives a corresponding large-magnitude negative penalty, . The Q-learning solution algorithm integrating ASA and ER The proposed approach enhances the standard Q-learning framework by embedding two complementary mechanisms: Adaptive Simulated Annealing (ASA) and Experience Replay (ER). In this architecture, the ASA strategy replaces the traditional static exploration policy to facilitate escape from local optima in damaged road segments. Simultaneously, the ER mechanism decouples the temporal correlation of rescue sequences by sampling past transitions from a memory buffer, ensuring stable policy convergence in high-uncertainty environments. The baseline Q-learning algorithm Q-learning is a classic model-free RL algorithm. Its objective is to learn the optimal action-value function, . The iterative update for its Q-table, derived from the Bellman optimality equation, is as follows: where represents the immediate comprehensive reward obtained by executing action in state . Additionally, ∈(0,1] is the learning rate that determines to what extent newly acquired information overrides old information, ∈[0,1) is the discount factor reflecting the importance of future rewards, represents the new state transitioned to, and denotes any possible action available in state . In the proposed formulation, this composite reward is mathematically defined as the algebraic sum of the individual reward components designed in the section “MDP formulation of the MCI rescue problem”, i.e., = + . This explicit integration ensures that the agent’s Q-value updating process is simultaneously driven by both the directional guidance towards the target and the imperative to avoid high-risk damaged road segments. The algorithm’s core decision-making mechanism, i.e., its action selection policy, typically employs the ε-greedy strategy: where rand(0,1) denotes a uniformly distributed random number generated between 0 and 1, and is the predefined exploration probability threshold. However, in the complex and high-risk scenario of MCI rescue, the standard Q-learning’s ε-greedy policy exposes its fundamental dilemmas, leading to two major challenges: Local Optimum Traps Induced by Risk Penalties: The ɛ-greedy strategy inherently relies on an uninformed stochastic selection between exploitation and random exploration. In the quantified MCI environment, the severe quadratic penalties from create deep ‘penalty basins’ in heavily damaged zones. Random exploration lacks the thermodynamic momentum to escape these steep basins, causing standard Q-learning to prematurely converge into local optima. Local Optimum Traps Induced by Risk Penalties: The ɛ-greedy strategy inherently relies on an uninformed stochastic selection between exploitation and random exploration. In the quantified MCI environment, the severe quadratic penalties from create deep ‘penalty basins’ in heavily damaged zones. Random exploration lacks the thermodynamic momentum to escape these steep basins, causing standard Q-learning to prematurely converge into local optima. Sparsity of Guidance Rewards: The widespread physical obstacles and damaged zones severely fragment the state space, causing the positive signals to become extremely sparse. Standard Q-learning updates are online and discard experiences immediately, failing to efficiently propagate these rare, successful terminal signals backwards. This sequential correlation results in high Q-value variance and prohibitively slow convergence. Sparsity of Guidance Rewards: The widespread physical obstacles and damaged zones severely fragment the state space, causing the positive signals to become extremely sparse. Standard Q-learning updates are online and discard experiences immediately, failing to efficiently propagate these rare, successful terminal signals backwards. This sequential correlation results in high Q-value variance and prohibitively slow convergence. To fundamentally address these environment-specific challenges, this paper integrates the ASA strategy to probabilistically escape the -induced traps, alongside the ER mechanism to globally propagate the sparse rewards. Exploration strategy optimization To address the propensity of the standard Q-learning algorithm and its ε-greedy policy to converge on local optima, this paper integrates the principles of SA to enhance the exploration mechanism. SA is a stochastic optimization algorithm based on the Monte Carlo method. Inspired by the physical annealing process in solid-state physics, it is highly effective at finding the global optimum for complex optimization problems. The core advantage of the SA algorithm lies in its intrinsic stochastic “jump” mechanism. This mechanism allows the algorithm to probabilistically accept a suboptimal solution (i.e., one that temporarily degrades the objective function) during its search, thereby enabling it to effectively avoid the pitfall of converging on a local optimum. In this research, we integrate the principles of SA with the Q-learning action selection process. At any given state , the agent no longer executes the ε-greedy policy. Instead, it adopts an action selection strategy based on the Metropolis acceptance criterion. The logic of this selection process is as follows: in state , the algorithm compares the Q-value of two candidate actions. First, the current best action is identified, = ,and its Q-value is stored as . Second, another candidate action, , is selected from the action space A. Its Q-value is stored as = . The algorithm’s decision to select (thereby potentially “jumping” away from the current best action) is governed by the following rules: If : The candidate action is superior to the current best. This new action is always accepted. If : The candidate action is superior to the current best. This new action is always accepted. If: The candidate action is suboptimal. It is still accepted with a probability P, which is calculated based on the Metropolis criterion: If: The candidate action is suboptimal. It is still accepted with a probability P, which is calculated based on the Metropolis criterion: Where is the temperature, which is dynamically annealed during the simulation. In the initial phase of the algorithm, the temperature is high. This results in a high probability of accepting suboptimal solutions, allowing the agent to conduct a broad exploration of the solution space. As the simulation progresses, is gradually decreased. This lowers the probability of accepting suboptimal moves, causing the algorithm’s behavior to converge on exploitation within the most promising regions. The temperature update process, or the cooling schedule, is a critical factor in determining the balance between global exploration and local refinement. Traditional cooling schedules, such as rigid exponential or logarithmic decay, often struggle to achieve timely convergence in the time-critical contexts of MCI rescue. To overcome this limitation, this study introduces a customized two-stage adaptive cooling strategy (Eqs. 6 and 7). Rather than relying on a uniform decay rate, this adaptive mechanism actively regulates the exploration-exploitation transition: it maintains a sustained high-temperature phase to ensure escape from hazard-induced local optima, followed by a rapid cooling phase to accelerate policy stabilization within the urgent rescue window. The specific temperature update rule is as follows: Here, represents the current episode, is the total number of iterations, is the initial temperature, is a tunable hyperparameter controlling the curvature of the cooling curve, and is a dynamic phase-weight coefficient determining the current cooling stage. Unlike traditional exponential or logarithmic decay functions, which operate independently of the environment’s penalty structure, the proposed cooling schedule (Eqs. 6 and 7) is a customized two-stage adaptive mechanism explicitly designed to counter the non-linear risk penalty defined in Eq. 2. During the initial phase ( = 1), the sustained high temperature ensures that the acceptance probability (Eq. 5) remains sufficiently large to bypass the deep local optima generated by severely damaged road segments ( < 1). In the subsequent phase ( = 1/t), the rapid temperature drop compels the algorithm to shift into pure exploitation. This specific adaptation differentiates the proposed approach from standard SA applications, directly aligning the exploration decay with the distinct topological constraints and strict time limits of MCI rescue. It is important to note that while the extremely rapid temperature drop in the second phase might theoretically raise concerns regarding premature convergence to a local optimum, this risk is effectively neutralized by the preceding high-temperature phase. Specifically, the initial 20% of episodes functions as a mandatory global exploration phase. This sustained high temperature provides sufficient exploratory momentum for the agent to bypass the deep, high-risk local traps characteristic of severely damaged road networks. Once a promising global basin of attraction is identified, the subsequent rapid cooling serves as a deliberate design choice tailored for MCI rescue scenarios. Compared to traditional logarithmic or exponential cooling schedules, which frequently expend critical computational time on redundant late-stage exploration, this rapid decay compels the algorithm to swiftly transition into pure exploitation. Consequently, it accelerates the stabilization of the optimal policy to meet the stringent temporal constraints of emergency response. Learning efficiency optimization To address the challenge of low learning efficiency in the standard Q-learning algorithm, this study incorporates the ER mechanism to optimize its learning process. While the ER mechanism is widely recognized as a cornerstone technique in Deep Reinforcement Learning to stabilize neural network training, its integration into tabular Q-learning is theoretically justified and highly beneficial for MCI scenarios. Although the Markov property guarantees that environmental state transitions are independent, the agent’s actual sampling trajectory consists of highly correlated sequential steps. In the context of damaged road networks with sparse and high-variance rewards, early successful trajectories to the MCI site are exceedingly rare and valuable. Standard tabular updates propagate these delayed rewards backward by only one step per episode. The ER mechanism acts as a background planning process, randomly sampling past transitions to break trajectory correlation and accelerating the backward propagation of delayed rewards across the tabular state space without requiring additional physical environment interactions. The Standard Q-learning algorithm employs an online learning approach: for every step the agent takes, it uses the generated experience transition (s, a, r, s’) to update the Q-table once. This experience is then immediately discarded. This use-and-discard model exhibits two major deficiencies when applied to complex problems: Low Sample Efficiency: Experiences are severely wasted. Many valuable—and often rare—successful experiences are utilized only a single time. Low Sample Efficiency: Experiences are severely wasted. Many valuable—and often rare—successful experiences are utilized only a single time. High Sample Correlation: Consecutive experiences are highly similar in time and space. This highly correlated sequence of training samples introduces bias and high variance into the Q-value updates. This, in turn, causes the learning process to oscillate violently and prevents the policy from converging stably. High Sample Correlation: Consecutive experiences are highly similar in time and space. This highly correlated sequence of training samples introduces bias and high variance into the Q-value updates. This, in turn, causes the learning process to oscillate violently and prevents the policy from converging stably. To counter these challenges, the ER mechanism creates a replay buffer D with a fixed capacity to store historical experiences. During training, the algorithm no longer learns solely from the most recent, real-time experience. Instead, it randomly samples a mini-batch of historical transitions from D to perform its updates. After introducing ER, the algorithm’s update process is adjusted as follows: Initialize a replay buffer D with capacity N Initialize a replay buffer D with capacity N At each time-step of an episode, the agent executes action in state , observing the reward and the next state . At each time-step of an episode, the agent executes action in state , observing the reward and the next state . Store the transition tuple () in the buffer D Store the transition tuple () in the buffer D Randomly sample a mini-batch of transitions ()from D () Randomly sample a mini-batch of transitions ()from D () For each transition in the mini-batch, compute the target Q-value, , based on the Bellman equation.8 For each transition in the mini-batch, compute the target Q-value, , based on the Bellman equation.8 Then, update the Q-table value using the learning rate Then, update the Q-table value using the learning rate While ER is conventionally employed in Deep Reinforcement Learning to stabilize neural network gradients, its integration into this tabular Q-learning framework serves a distinct, problem-specific purpose. In severely fragmented MCI road networks, the terminal path guidance reward is extremely sparse. By shifting from a sequential, single-step update to a randomized batch update (Eq. 9), this modification decouples the learning process from the agent’s immediate physical trajectory. This specifically accelerates the backward propagation of rare, successful terminal signals across the disconnected grid states, thereby addressing the low sample efficiency inherent in standard tabular Q-learning when applied to disaster scenarios. Specifically, the ER mechanism enhances the accuracy of action selection in the current state through the global propagation of Q-values. By continuously sampling and learning from diverse historical transitions that are not temporally or spatially adjacent to the current position, the algorithm effectively propagates delayed rewards across the broader state space. Consequently, when the agent evaluates candidate actions in its current state, the corresponding Q-values have already been refined by these historical updates. This provides a more accurate and unbiased estimation of the long-term expected return, thereby preventing the agent from making suboptimal action selections based merely on short-sighted local evaluations. Experiments and results To systematically validate the efficacy of the proposed MCI decision model and the hybrid algorithm, this chapter presents a series of comparative simulation experiments. This chapter is organized as follows: We first introduce the experimental setup, including the environment parameters and the evaluation metrics. Subsequently, we benchmark our proposed method against several baseline algorithms and analyze the results of an ablation study. Finally, the algorithm’s performance is comprehensively evaluated from two perspectives: quantitative metrics and visualized pathing results. Experimental setup In this study, we constructed a 20 × 20 grid map, as illustrated in Fig. 4. Within this environment, white cells represent normal, traversable paths that provide a standard movement reward, while black cells represent impassable obstacles which the agent cannot enter. Furthermore, the blue cell indicates the agent’s starting point (Start), and the yellow cell indicates the goal. Most importantly, red cells of varying shades represent roads with different degrees of degradation; the darker the shade, the more severe the road’s degradation and the higher its associated traversal risk. The 20 × 20 experimental grid environment. To address the procedural specifics and ensure the rigorousness and reproducibility of the experiments, it is crucial to clarify that the test environments are procedurally generated fixed benchmarks controlled by specific random seeds. The environment generation protocol consists of two sequential algorithms: Base Topology Generation: A predefined number of impassable obstacles are randomly distributed across the grid using a unified base random seed. Subsequently, a Breadth-First Search (BFS) validation algorithm is strictly employed to ensure that at least one viable path exists between the start point and the MCI site, preventing the generation of unsolvable dead-end maps. Base Topology Generation: A predefined number of impassable obstacles are randomly distributed across the grid using a unified base random seed. Subsequently, a Breadth-First Search (BFS) validation algorithm is strictly employed to ensure that at least one viable path exists between the start point and the MCI site, preventing the generation of unsolvable dead-end maps. Seismic Damage Injection: Based on the specific seismic impact level (1 to 5), probabilistic damage is applied to the traversable cells, primarily concentrated within a specific radius around the target MCI site. Higher seismic levels correlate with higher probability weights for generating severe damage. A distinct random seed is uniquely assigned to each of the five seismic levels to control this stochastic degradation process. Following the damage injection, the BFS validation is re-executed to guarantee the theoretical solvability of the degraded map. Seismic Damage Injection: Based on the specific seismic impact level (1 to 5), probabilistic damage is applied to the traversable cells, primarily concentrated within a specific radius around the target MCI site. Higher seismic levels correlate with higher probability weights for generating severe damage. A distinct random seed is uniquely assigned to each of the five seismic levels to control this stochastic degradation process. Following the damage injection, the BFS validation is re-executed to guarantee the theoretical solvability of the degraded map. By explicitly locking the random seeds for both the base topological structure and the level-specific damage distributions, we effectively transformed the randomly generated scenarios into fixed, standardized benchmarks. This strict protocol guarantees that all comparative algorithms (baseline, ablated, and proposed) evaluated in this study were tested on the exact same environmental instances, thereby ensuring the absolute fairness and statistical validity of the performance comparisons. The algorithms selected for comparative analysis include standard Q-learning, several ablated algorithms, and our final hybrid algorithm. The specific hyperparameters for these algorithms are summarized in Table 3. Table 3Summary of Experimental Hyperparameters.ParameterSymbolValueDescriptionMap SizeN×N20 × 20Dimensions of the grid mapTraining IterationsZ500Total number of training episodesDiscount Factorγ0.8Weighting for future rewardsLearning Rate 0.8Fixed learning rate for standard Q-learning.Improved Algorithm ParametersInitial Temperature 500Initial temperature for the proposed algorithmCooling Parameterξ2Controls the curvature of the cooling scheduleBaseline Algorithm ParametersExploration Rateε0.3,0.5,0.8Exploration rate for the ε-greedy policy. Summary of Experimental Hyperparameters. To ensure a fair and rigorous comparison, all subsequent experiments evaluating the baseline, ablated, and proposed algorithms (sections “Baseline algorithm performance analysis” to “Analysis of environmental adaptability and scalability”) were strictly conducted within the identical 20 × 20 quantified grid environment defined above, with exactly the same obstacle distribution and damage levels. Furthermore, to mitigate the impact of stochastic exploration inherent in reinforcement learning and to ensure the statistical consistency of the comparative results, the performance metrics for each algorithm were averaged over multiple independent training runs under identical initial conditions. Baseline algorithm performance analysis To evaluate the performance of the standard Q-learning algorithm in MCI scenarios, we first benchmarked its performance under different exploration rates. As illustrated in Fig. 5, with a low exploration rate (ε = 0.3), the standard Q-learning algorithm converges rapidly and finds a relatively short path to the goal. However, the average Q-value remains exceptionally low. This is because the agent in this state has prematurely ceased exploring other unknown, broader regions of the map. Consequently, the agent is highly prone to becoming trapped in a local optimum, failing to discover the globally optimal path. Performance of standard Q-learning in the MCI Scenario. Conversely, a high exploration rate (ε = 0.8), while significantly increasing the overall Q-table values (indicating more thorough exploration), causes the algorithm’s step count per episode to remain highly unstable and persistently high. This prevents the formation of an efficient, converged policy. This exposes the fundamental shortcoming of standard Q-learning in complex environments: its inability to simultaneously achieve learning efficiency, policy stability exploration. Ablation study and performance comparison Ablation study This section presents an ablation study designed to validate two key points: (1) the individual efficacy of the ER and ASA modules, and (2) the synergistic performance gains achieved when they are combined. To ensure the transparency and reproducibility of the reinforcement learning environment, the exact values of the key reward coefficients are explicitly defined as follows: Goal Reward (): +100, awarded when the agent successfully reaches the target node. Goal Reward (): +100, awarded when the agent successfully reaches the target node. Collision Penalty (): −200, applied and terminating the current episode if the agent hits a boundary or an absolute obstacle. Collision Penalty (): −200, applied and terminating the current episode if the agent hits a boundary or an absolute obstacle. Step Penalty: −1 per step, encouraging the discovery of shorter paths. Step Penalty: −1 per step, encouraging the discovery of shorter paths. Risk Penalty: The coefficient for the risk penalty is set to −50, yielding = −50 ×, where represents the RCR ranging from 0 to 1. Risk Penalty: The coefficient for the risk penalty is set to −50, yielding = −50 ×, where represents the RCR ranging from 0 to 1. Furthermore, the implementation details of the ER mechanism are critical for algorithmic reproducibility. In our proposed ER-ASA framework, the replay buffer size is set to N = 2000 transitions, ensuring a sufficient memory pool of historical interactions without excessive computational overhead. During each learning step, a mini-batch of size B = 64 is randomly sampled from the buffer to update the Q-values, thereby effectively breaking the temporal correlations among consecutive path-finding sequences. As illustrated in Figs. 6, 7 and 8, the introduction of ER does not, by itself, show a significant difference in the final converged step count. However, its primary impact is clearly visible in the Q-table metrics. The rapid ascent and higher final plateau of the average Q-value corroborate that ER—by enhancing the utilization of critical experiences and breaking sample correlation—enables the agent to learn a higher-quality policy and thus discover superior paths. Comparison Results (ε = 0.3). Comparison Results (ε = 0.5). Comparison Results (ε = 0.8). To validate the independent efficacy of the ASA exploration strategy, this experiment benchmarked it against the Standard Q-learning method (Fig. 9). The reward curve for the ASA-only algorithm ascends at the fastest rate and converges to a stable, high-level plateau. This indicates that the strategy successfully discovered a superior-quality global optimum. In terms of average step count, the ASA algorithm exhibits greater stability than the baseline after its initial rapid convergence. These results strongly corroborate that the ASA strategy, via its intrinsic probabilistic “jump” mechanism, effectively overcomes the propensity of the ε-greedy policy to become trapped in local optima, achieving significant improvements in both policy quality and the smoothness of the learning process. Performance Comparison: ASA-only Algorithm. Finally, the two improvements were combined to form the complete hybrid algorithm (Fig. 10). Analyzing both the average reward and average step curves, the hybrid algorithm outperforms all other tested methods. It not only achieves the fastest convergence rate but also stabilizes at the highest reward plateau and the lowest average step count. Notably, when compared to the runner-up (the ASA-only algorithm), the hybrid method’s reward curve ascends even more rapidly. This highlights the significant performance gains contributed by the sample efficiency of the ER mechanism. Performance Comparison: ER + ASA. Sensitivity analysis of cooling schedules To evaluate the impact of different temperature update functions on the algorithm’s convergence performance and global exploration capabilities, a sensitivity analysis was conducted. The proposed ASA strategy was benchmarked against two widely adopted conventional simulated annealing decay functions: exponential decay and logarithmic decay, under identical experimental conditions. As illustrated in Fig. 11, the varying cooling schedules exhibit distinctly different convergence behaviors in the complex MCI environment. The exponential decay schedule features a rapid initial temperature drop, which causes the agent to prematurely transition into the exploitation phase. Consequently, the rapid loss of exploration momentum leads the algorithm to converge to a local optimum, evidenced by a suboptimal average reward and a persistently high step count. Conversely, the logarithmic decay schedule maintains a higher temperature over a prolonged period. While this prolonged exploration prevents early convergence to local optima, it induces continuous random actions during the later stages of training. This results in significant performance oscillations and an inability to stabilize an optimal path within the required timeframe. Sensitivity analysis of different cooling schedules. In contrast, the proposed ASA algorithm utilizes an initial sustained high-temperature phase to effectively bypass high-risk local traps. The subsequent rapid cooling phase compels a swift transition to pure exploitation. The empirical results demonstrate that this two-stage mechanism enables the ASA algorithm to converge to the highest average reward and the lowest step count. This indicates that the customized decay strategy successfully accelerates policy stabilization while preserving global optimality, rendering it highly suitable for time-critical post-disaster routing tasks. Algorithm performance and convergence analysis To comprehensively evaluate the search efficiency and global optimization capabilities of the proposed ER-ASA framework, this section benchmarks it against three representative algorithms under identical experimental conditions. The selected baselines—SARSA16, Standard Q-learning, and the HSA-QL algorithm27—provide a rigorous comparative spectrum ranging from classic reinforcement learning architectures to modern heuristic-enhanced methods. To capture the representative learning behavior and mitigate the impact of individual stochastic fluctuations, the performance metrics for each algorithm were derived from multiple independent runs under the MCI Level 3 damage intensity. Figure 12 illustrates the dynamic evolution of movement steps and cumulative rewards for these algorithms during the training process. Performance Comparison of Four Algorithms. Regarding step consumption, reveals distinct convergence behaviors. Standard Q-learning and SARSA exhibit gradual and erratic descents in the initial stages due to their reliance on undirected random exploration. Interestingly, the HSA-QL algorithm, despite its heuristic guidance, maintains consistently high step counts and struggles with persistent oscillations. This indicates that in fragmented disaster topologies with non-linear penalty spaces, generic distance-based heuristics often lead to severe search redundancy. In contrast, the proposed ER-ASA algorithm demonstrates a more direct and efficient descent to the lowest step-count range. Benefiting from the Experience Replay mechanism, the agent efficiently breaks temporal correlations between samples, rapidly identifying high-efficiency trajectories with minimal environmental interaction costs. A critical distinction among the evaluated algorithms emerges in the reward acquisition curves, which reflect their underlying hazard-avoidance mechanisms. Due to undirected exploration, Standard Q-learning and SARSA consistently yield suboptimal reward levels, as their highly random action selection frequently results in boundary collisions and early episode terminations. Furthermore, the HSA-QL algorithm exhibits a severe reward deficit, plunging to the extreme negative lower bound of the chart. This performance degradation occurs because its Euclidean distance heuristic rigidly directs the agent toward the target, forcing it into severe structural damage zones. Without sufficient exploration momentum to escape these high-penalty regions, the HSA-QL agent remains trapped in local optima, accumulating continuous step-wise penalties. In contrast, the reward curve of the ER-ASA algorithm demonstrates a steady ascent, ultimately stabilizing at a high positive plateau. By replacing rigid heuristic guidance with a customized two-stage adaptive cooling strategy, the proposed agent sustains adequate thermodynamic momentum to actively detour around deep penalty basins. These results substantiate that the ER-ASA framework successfully balances exploration and safety, securing optimal trajectories in high-risk, fragmented environments where conventional distance heuristics fail. Analysis of environmental adaptability and scalability Adaptability analysis To validate the adaptive decision-making capabilities of the final hybrid algorithm in environments with varying risk levels, we simulated its application in MCI scenarios across five different seismic impact levels. The resulting optimal paths are illustrated in Fig. 13. The results demonstrate that the algorithm successfully planned a viable path to the MCI site in all scenarios. Crucially, the path-decision logic exhibited a high degree of environmental adaptability: in environments with minor degradation, the algorithm favored efficiency by selecting shortcuts that traversed low-risk zones, whereas in severely degraded environments, it intelligently identified and completely bypassed large high-risk core zones, charting a longer, but demonstrably safer, detour. Visualization of the Algorithm’s Adaptive Pathing in Environments with Varying Degradation Levels. Scalability analysis MCI emergency response scenarios vary significantly in geographical scale, ranging from neighborhood blocks to entire urban districts. Therefore, an effective decision algorithm must not only adapt to varying risk levels but must also possess scalability to handle the dramatic increase in state space and the associated computational burden posed by larger maps. To isolate and validate the scalability of our proposed hybrid algorithm, we standardized the environmental complexity and then tested the algorithm’s routing performance on two larger-scale maps: 30 × 30 (Fig. 14) and 40 × 40 (Fig. 15). Path Planning Result on the 30 × 30 Map. Path Planning Result on the 40 × 40 Map. As this experiment confirms, our proposed hybrid algorithm exhibits robust scalability. Instead of succumbing to the curse of dimensionality or failing to converge amid the dramatically larger state space, the algorithm consistently planned high-quality rescue paths that balanced both safety and efficiency. This property provides a viable foundation for the algorithm’s deployment in real-world MCI scenarios of varying scales. Online re-planning in response to secondary structural changes In real-world MCI scenarios, post-disaster environments are rarely static. Secondary disasters, such as aftershocks, can instantly alter road connectivity, rendering previously optimal routes impassable. To evaluate the algorithm’s responsiveness and online re-planning capabilities in such dynamic settings, a secondary disruption experiment was conducted. The primary objective of this experiment is to simulate a secondary disaster scenario. In a post-disaster rescue context, the road network is not only statically damaged but also susceptible to further structural collapses. This test evaluates the agent’s ability to rapidly sense environmental disruptions and execute real-time path adjustments without a complete restart of the training process. In this simulation, the agent first underwent a pre-training phase on a Level 3 damaged map to establish a baseline optimal policy. It should be explicitly noted that, due to the complexity of the dynamic scenario, the total number of training episodes for this specific experiment was extended from the standard 500 to 1000. This extension was implemented to fully observe the algorithm’s stable convergence both before and after the disruption. Specifically, a simulated aftershock was triggered at episode 600, instantly altering the local road connectivity. The dynamic recovery curve in Fig. 17 illustrates the entire process: the agent maintains a converged optimal trajectory prior to the event, experiences a sudden environmental invalidation at episode 600, and subsequently executes rapid online re-planning to discover a new safe route. Dynamic recovery curve demonstrating the rapid adaptation of the proposed algorithm following a sudden map update. The algorithmic response to this dynamic topological change is visualized sequentially in Fig. 17. Initially (Fig. 17a), the agent progresses along the pre-planned route. Upon the occurrence of the aftershock (Fig. 16b), the sudden degradation of local road connectivity blocks the original path, leading to an inevitable collision (marked by a red ‘X’). This illustrates the vulnerability of static pre-planning to dynamic environments. However, by leveraging the ER mechanism, the proposed algorithm immediately initiates online re-planning. It updates the local Q-values based on the newly encountered severe penalties and re-evaluates global historical transitions. Consequently, the agent rapidly abandons the blocked route and discovers a new, safe global detour (Fig. 16c). Online re-planning under dynamic road damage. a Initial planned route. b Route blocked by sudden RCR degradation. c New global detour after re-planning. Furthermore, Fig. 17 quantitatively illustrates the dynamic recovery performance during this event. Prior to the disruption, the algorithm consistently converges to the optimal Manhattan distance. At episode 600, the sudden map update invalidates the original route, causing a sharp, momentary spike in the average steps as the agent navigates the newly formed hazards. Owing to the ER mechanism’s efficient reuse of global experiences, the algorithm rapidly converges back to the original minimum step count within approximately 20 episodes. This rapid recovery to the identical minimum step count demonstrates that the agent successfully discovered a new detour that entirely circumvents the disaster zone while maintaining the optimal Manhattan distance. The results objectively confirm that the proposed algorithm can acutely capture real-time changes in RCR parameters and execute rapid global re-planning without incurring additional traversal costs. Conclusion This paper addresses the critical challenge of emergency path planning following MCIs, a problem characterized by severe road network damage and high environmental uncertainty, by proposing a novel decision-support algorithm. To overcome the inherent limitations of standard Q-learning—specifically its difficulty in balancing exploration efficiency with convergence speed, its propensity to fall into local optimum traps, and its sluggish learning process—we propose an improved Q-learning algorithm that integrates an ER mechanism with an ASA exploration strategy. Simulation results strongly validate the efficacy of the proposed method. The improved Q-learning algorithm demonstrated significant performance gains; compared to both the high-performing baseline algorithms and the safer, more reliable SARSA algorithm, agents guided by the improved algorithm achieved higher reward values. This indicates the capability to identify paths with lower traversal costs, thereby providing rescue teams with faster and more reliable routing solutions. Despite these positive results, this study has certain limitations, which also highlight directions for future research. First, the current environment modeling is based on grid maps, which simplifies the complex topological structure of real-world urban road networks. Future research should focus on migrating and applying this algorithm to graph-based real-world road network data. Second, this study currently considers only single-agent path planning, whereas real-world MCI response requires multi-party coordination. Therefore, extending this framework into a Multi-Agent Reinforcement Learning system to achieve coordinated scheduling of rescue fleets represents a highly valuable research direction. Finally, while this study introduced a simulated aftershock to validate the algorithm’s online re-planning capabilities, this localized disruption remains a simplified abstraction. Real-world MCI environments are characterized by compounding, multi-variable uncertainties. Therefore, future work should incorporate continuous and large-scale dynamic factors—such as cascading aftershocks, evolving traffic congestion, and asynchronous road repair progress—to evaluate the algorithm’s real-time adaptability in truly complex, high-fidelity disaster scenarios. Author contributions Sheng Wang conceptualized the study, developed the methodology, created the visualizations, conducted the investigation, and wrote the original draft. Yang Peng acquired the funding, administered the project, provided supervision and resources, and contributed to writing (review and editing). Jingrong Yang curated the data, validated the results, provided supervision, provided resources, and contributed to writing (review and editing). Funding This work is supported by National Key Research and Development Program of China under Grant no. 2023YFC3807402-02. Data availability The datasets generated during or analysed during the current study are available from the corresponding author on reasonable request. Declarations Competing interests The authors declare no competing interests. Footnotes Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. References Associated Data Data Availability Statement The datasets generated during or analysed during the current study are available from the corresponding author on reasonable request.