Factorized Deep Q-Network for Cooperative Multi-Agent Reinforcement Learning in Victim Tagging

Maria Ana Cardei, Afsaneh Doryab · 2025

Read Factorized Deep Q-Network for Cooperative Multi-Agent Reinforcement Learning in Victim Tagging by Maria Ana Cardei, Afsaneh Doryab (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-00684.txt.

Introduction

Maria Ana Cardei and Afsaneh Doryab wrote a 2025 paper on speeding up the tagging of victims after mass casualty events. In such chaos, responders must quickly mark each victim's condition so later care can be prioritized. The tagging step must be fast, and every minute shapes what follows. The authors frame the task as a team problem for many responders acting at once.

They built five practical rules responders could follow, differing in how much each responder knows: only local sightings, or a shared global picture. Simulations showed local rules did better, above all a rule sending each responder to the nearest untagged victim, with freedom to replan. They then trained a learning system whose agents share one value network split into parts. The learning approach won in smaller scenes. The hand rules won as scenes grew more complex.

Catalog record

CREATORMaria Ana Cardei, Afsaneh Doryab
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE63 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2503-00684

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Factorized Deep Q-Network for Cooperative Multi-Agent Reinforcement Learning in Victim Tagging Maria Ana Cardei, Afsaneh Doryab Abstract Mass casualty incidents (MCIs) are a growing concern, characterized by complexity and uncertainty that demand adaptive decision-making strategies. The victim tagging step in the emergency medical response must be completed quickly and is crucial for providing information to guide subsequent time-constrained response actions. In this paper, we present a mathematical formulation of multi-agent victim tagging to minimize the time it takes for responders to tag all victims. Five distributed heuristics are formulated and evaluated with simulation experiments. The heuristics considered are on-the go, practical solutions that represent varying levels of situational uncertainty in the form of global or local communication capabilities, showcasing practical constraints. We further investigate the performance of a multi-agent reinforcement learning (MARL) strategy, factorized deep Q-network (FDQN), to minimize victim tagging time as compared to baseline heuristics. Extensive simulations demonstrate that between the heuristics, methods with local communication are more efficient for adaptive victim tagging, specifically choosing the nearest victim with the option to replan. Analyzing all experiments, we find that our FDQN approach outperforms heuristics in smaller-scale scenarios, while heuristics excel in more complex scenarios. Our experiments contain diverse complexities that explore the upper limits of MARL capabilities for real-world applications and reveal key insights. I Introduction Mass casualty incidents (MCIs) are defined as situations in which casualties greatly outnumber available local resources, which eventually overwhelms the local healthcare system within a time frame [ 1 ] . Some common types of MCIs involve natural disasters such as wildfires, hurricanes, and earthquakes or terrorist attacks such as mass shootings and bombings. When emergency medical responders arrive at the scene of an MCI, an initial task is to locate individuals and assess their injuries promptly, known as triage, followed by physically tagging them with a color-coded label corresponding to injury severity. All victims are tagged before assessing the next steps. Fast and efficient victim tagging is a critical initial stage in the MCI response process, providing vital information that guides subsequent decisions. It enables the timely assessment of the total number of victims, their distribution based on the severity of injuries, and their respective locations throughout the MCI environment. Most of the current work to improve the emergency response to MCIs focuses on improving resource allocation [ 2 ] , responder coordination [ 3 , 4 ] , or transportation to treatment facilities [ 5 ] . In [ 6 ] , authors model the rescue, treatment, and transportation of MCI victims to hospital locations, testing communication modes between agents. In [ 7 ] , authors suggest that timing is a critical factor in saving lives and discuss the triage process; however, similar to other work, they overlook and assume the tagging process. There is a lack of research done in the on-scene or pre-hospitalization stage, specifically the task of tagging victims fast. To our knowledge, the problem of minimizing victim tagging time has not been formally addressed. Therefore, we leverage insights from MCI response and similar fields, such as search and rescue, to compare and evaluate efficient, practical heuristics for this problem. Our goal is to develop insights that help emergency departments refine MCI planning procedures and enhance the coordination of human or hybrid teams through more effective victim tagging strategies. Multi-agent reinforcement learning (MARL) has proven effective in a range of domains, such as emergency response and collaborative robotics, because it naturally handles decentralized decision-making under uncertainty [ 8 , 9 , 10 ] . Existing MARL research for disaster response, such as fire evacuation or patient admission, typically restricts environments to smaller grids (e.g., 25 \times 25 in [ 10 ] ) or fewer agents (e.g., up to 100 in [ 9 ] ). Nonetheless, these studies highlight the promise of using reinforcement learning (RL) for complex, evolving tasks like victim tagging. In this paper, we study the victim tagging problem for a team of medical responders. We formalize victim tagging as an integer linear programming problem and evaluate and compare five distributed, on-the-go responder heuristics to minimize the time it takes to tag all victims under varying levels of uncertainty in communication. We further investigate the performance of a MARL method, F actorized D eep Q-N etwork, which we call ”FDQN”, to minimize victim tagging time, as compared with the heuristic baselines. Our results show that, among the heuristics, the Local Nearest Victim Policy outperforms others, and local policies (under greater uncertainty) are more efficient than global policies for adaptively choosing the next victim to tag. Individual heuristic policy analyses give us additional insights into the number of responders needed based on the number of victims and the efficiency of each policy, measured in the time it takes to tag victims. Our evaluation also shows that the proposed FDQN outperforms heuristic baselines in simpler scenarios with lower responder-to-victim ratios and in smaller MCI environments, while heuristics perform better in more complex settings. These results provide valuable insights both for MARL researchers pursuing more realistic, large‐scale applications and for emergency management departments seeking to improve their decision-support strategies. The contributions of this paper are summarized as follows: 1. We present the first integer linear programming (ILP) formulation of the MCI victim tagging task. We expose its combinatorial complexity and use the formulation as a baseline to demonstrate our heuristics and multi-agent reinforcement learning method. 2. We design five distributed, on-the-go heuristic solutions considering global and local agent communication and analyze their performance through simulations. Results indicate that a local, nearest victim tagging solution minimizes victim tagging time in uncertain situations, as compared to four other practical heuristics. 3. We design and implement a factorized deep Q-network (FDQN) featuring a shared state space, decentralized decision-making, and constraint-based action pruning for cooperative multi-agent victim tagging. We evaluate its performance against baseline heuristics through extensive, scalable simulation experiments. Results show that FDQN outperformed heuristics in smaller‐scale MCIs and shows promise for scaling to larger environments and higher agent counts. Our experiments have varying scales, some with expansive areas and sizeable agent teams, to explore upper limits of performance, thus offering key insights into real-world MARL applications. This article is an extended version of a previous conference paper [ 11 ] . In this journal article, we propose the use of FDQN to learn optimal responder team policies for efficient victim tagging in an MCI emergency medical response. We implemented FDQN and conducted additional experiments, involving our RL strategy integrated with the multi-agent system, as well as additional heuristic-based simulations for comparison. Finally, more technical details and discussions have been added. II Related Work We first survey problems and methods that share parallels with victim tagging and highlight why existing solutions do not fully address the demands of real‐time, uncertain multi‐agent environments. We then explore MARL as an adaptive approach suited to these challenges. II-A Related Problems and Methods II-A 1 Search and Rescue (SAR) as Task Allocation Victim tagging has not been widely studied, so we look to SAR literature for similar multi-agent decision-making scenarios. Many studies evaluate SAR as a task allocation problem, considering a team of unmanned aerial vehicles (UAVs), hybrid vehicles [ 12 , 13 , 14

Read the complete text