Vulcan : Vision-Language-Model Enhanced Multi-Agent Cooperative Navigation for Indoor Fire-Disaster Response

Shengding Liu, Qiben Yan · 2026

Read Vulcan : Vision-Language-Model Enhanced Multi-Agent Cooperative Navigation for Indoor Fire-Disaster Response by Shengding Liu, Qiben Yan (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-12831.txt.

Introduction

Shengding Liu and Qiben Yan build VULCAN, a multi-agent navigation system for indoor fire rescue. Existing vision-based systems suit calm indoor settings and fail badly as fire spreads. VULCAN pairs multi-modal perception with vision-language models for hazard-aware planning. The team extends Habitat-Matterport3D with realistic smoke, heat, and sensor damage. Baselines are tested in normal and fire-driven settings. Results expose failure modes of current methods under fire and stress robust perception for reliable rescue.

Catalog record

CREATORShengding Liu, Qiben Yan
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE25888
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.12831

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Vulcan : Vision-Language-Model Enhanced Multi-Agent Cooperative Navigation for Indoor Fire-Disaster Response Shengding Liu, Qiben Yan Abstract Indoor fire disasters pose severe challenges to autonomous search and rescue due to dense smoke, high temperatures, and dynamically evolving indoor environments. In such time-critical scenarios, multi-agent cooperative navigation is particularly useful, as it enables faster and broader exploration than single-agent approaches. However, existing multi-agent navigation systems are primarily vision-based and designed for benign indoor settings, leading to significant performance degradation under fire-driven dynamic conditions. In this paper, we present Vulcan , a multi-agent cooperative navigation framework based on multi-modal perception and vision–language models (VLMs), tailored for indoor fire disaster response. We extend the Habitat-Matterport3D benchmark by simulating physically realistic fire scenarios, including smoke diffusion, thermal hazards, and sensor degradation. We evaluate representative multi-agent cooperative navigation baselines under both normal and fire-driven environments. Our results reveal critical failure modes of existing methods in fire scenarios and underscore the necessity of robust perception and hazard-aware planning for reliable multi-agent search and rescue. I Introduction Natural and human-made disasters pose immediate threats to life and infrastructure, requiring rapid and coordinated emergency response. In hazardous environments such as indoor fires, high temperature, dense smoke, and structural instability make human intervention extremely dangerous. A recent devastating fire in Tai Po, Hong Kong [ 1 ] greatly impeded human access and situational awareness during the early response phase, leading to significant casualties and property damage. It was the deadliest fire in Hong Kong in decades. Mobile robots have emerged as a promising alternative for operating in life-threatening environments. By enabling autonomous exploration, hazard perception, and target search under severe perception degradation, robotic systems can provide rapid situational awareness while reducing the risk to human responders and supporting more informed decision-making. Despite recent progress, most existing search and rescue (SAR) deployments [ 5 ] still rely on single-robot exploration or loosely coordinated multi-robot routines, where individual robots operate with minimal information sharing. Such strategies inherently limit system-level efficiency, particularly in large, unmapped, or dynamically evolving indoor environments, thereby underscoring the urgent need for multi-agent cooperative navigation. Fig. 1 shows an example scenario of multi-agent search and rescue in an indoor fire environment. Fig. 1: Overview of Vulcan for multi-agent search and rescue in indoor fire scenarios. (a) RGB camera (b) Depth camera (c) Lidar (d) Thermal camera Fig. 2: Gazebo-based fire simulation demonstrates the degradation of modality-dependent perception as smoke density increases. In recent years, vision-based cooperative navigation has attracted increasing attention, driven by advances in simulation platforms, and large-scale 3D scene datasets [ 6 ] . Existing approaches to visual navigation are typically grouped into two main paradigms: end-to-end systems [ 3 ] , which directly map sensory observations to control actions, and modular pipelines [ 7 ] , which explicitly decompose the task into perception, mapping, planning, and control. Both paradigms have demonstrated effectiveness by leveraging spatial and semantic representations learned via supervised or reinforcement learning. Furthermore, recent work has increasingly explored the integration of vision–language models (VLMs) into visual navigation systems [ 10 ] . By jointly reasoning over visual observations and natural language, VLMs introduce powerful semantic priors that enhance object grounding, relational reasoning, and holistic understanding of complex indoor environment. However, the majority of existing multi-agent cooperative navigation frameworks [ 10 , 4 , 11 ] are predominantly vision-based and developed under assumptions of static, well-structured, and relatively benign indoor environments. Such assumptions fundamentally break down in fire-driven indoor scenarios, where perception is severely degraded by smoke and high temperatures, environmental conditions evolve rapidly and unpredictably, and inefficient or uncoordinated exploration can directly compromise both agent safety and mission success. These problems highlight a critical gap in current navigation frameworks and motivate the need for an efficient and safe cooperative multi-agent navigation tailored for indoor fire-disaster response. Fig. 3: System architecture of Vulcan . Each agent performs multi-modal perception and fusion, constructs hazard-aware global maps, and plans safe and efficient exploration using a VLM-based global planner and hazard-aware FMM local planner. Despite recent advancements in cooperative navigation including learning-based approaches [ 3 ] , VLM powered planning [ 10 ] , and embodied multi-agent coordination frameworks [ 8 ] , directly applying these approaches to fire-driven indoor scenarios remains extremely challenging. Compared to static or benign environments, indoor fire scenes introduce several fundamental challenges: I- 1 Dynamic and rapidly evolving environments Temperature gradients, flame propagation, and smoke diffusion continuously reshape the environment, altering traversability, visibility, and semantic cues over time. I- 2 Unreliable and degraded sensor data Fire-induced smoke severely degrades multi-modal perception in indoor environments. As illustrated in Fig. 2 , our Gazebo-based fire simulations, built on a high-fidelity physics engine with particle-emitter support, qualitatively demonstrate modality-dependent perception degradation under increasing smoke density. I- 3 Latency-critical and hazard aware decision making SAR missions demand real-time coordination, as excessive planning delays or inefficient multi-robot routing can directly compromise human survival and overall mission success. Simultaneously, it is crucial to ensure that each agent can navigate and explore safely, avoiding hazards while completing its tasks. To address these challenges, we introduce Vulcan 1 1 1 Vulcan is the Roman deity of fire. , a hazard-aware multi-agent cooperative navigation framework tailored for indoor fire SAR scenarios. As illustrated in Fig. 1 , each agent performs multi-modal perception and fusion to jointly construct a hazard-aware global map. Based on this shared representation, exploration is coordinated through a VLM-based global planner and a hazard-aware local planner to ensure safe and efficient navigation. While a complete end-to-end deployment is ongoing, we implement key components and present preliminary evaluations demonstrating the framework’s feasibility and the limitations of conventional multi-agent navigation in fire scenarios. Live demonstrations and representative results are available at https://vulcan-dr.github.io . To summarize, our contributions are as follows: • We propose Vulcan , a hazard-aware multi-agent cooperative navigation framework tailored for indoor fire scenarios, which maintains high navigation efficiency despite severe fire-induced perception degradation and environmental hazards. • We extend indoor navigation benchmarks with realistic fire simulations and coordinate a VLM-based global planner with a hazard-aware local controller. • We evaluate representative multi-agent navigation baselines under both normal and simulated indoor fire conditions, and systematically analyze the underlying causes of their performance degradation in fire scenarios. II System Design II-A Task Definition We consider a multi-agent cooperative SAR task in fire-driven indoor environments, where a team of mobile robots explores an unknown scene to locate victims or mission-critical targets under hazardous conditions. Let \mathcal{C}=\{c_{1},\ldots,c_{m}\} denote the set of target categories (e.g., human , fire extinguisher , bathroom ), and let \mathcal{S}=\{s_{1},\ldots,s_{K}\} denote the set of fire-affected indoor scenes. In each episode, N robots \mathcal{R}=\{r_{1},\ldots,r_{N}\} are deployed in a scene s_{i}\in\mathcal{S} from a shared starting position \mathbf{p}_{i} with different initial orientations to reflect realistic emergency deployment. All robots are assigned the same target category c_{i}\in\mathcal{C} . The episode is defined as: \mathcal{T}_{i}=\{N,s_{i},c_{i},\mathbf{p}_{i}\}. (1) The observation space \mathcal{O} comprises four modalities: RGB, depth, thermal imaging, and mmWave radar, providing complementary visual, geometric, and smoke-robust perception. The action space \mathcal{A} consists of six discrete actions: { move_forward , turn_left , turn_right , look_up , look_down , stop }. The move_forward action advances the robot by 0.25\,\text{m} , and each rotational action changes

Read the complete text