RAVEN: Resilient Aerial Navigation via Open-Set Semantic Memory and Behavior Adaptation
Seungchan Kim, Omar Alama, Dmytro Kurdydyk, John Keller, Nikhil Keetha, Wenshan Wang, Yonatan Bisk, Sebastian Scherer · 2025
Read RAVEN: Resilient Aerial Navigation via Open-Set Semantic Memory and Behavior Adaptation by Seungchan Kim, Omar Alama, Dmytro Kurdydyk, John Keller, Nikhil Keetha, Wenshan Wang, Yonatan Bisk, Sebastian Scherer (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-23563.txt.
Introduction
A drone hunting for a lost object outdoors cannot rely on room layouts or fixed maps. Seungchan Kim and 7 co-authors present RAVEN, a navigation system for aerial robots in open, unstructured places. It keeps a 3D semantic memory built from voxels and rays, so the robot plans over long horizons instead of reacting to each view. Short-range voxel search and long-range ray search let it scale to big areas.
A vision-language model suggests extra cues when targets are scarce. A behavior tree switches strategies to keep the search robust. Across ten simulated outdoor scenes and one hundred tasks, RAVEN beat baselines by a wide margin, and field tests on a real drone followed.
Catalog record
| CREATOR | Seungchan Kim, Omar Alama, Dmytro Kurdydyk, John Keller, Nikhil Keetha, Wenshan Wang, Yonatan Bisk, Sebastian Scherer |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 48 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2509-23563
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
RAVEN: Resilient Aerial Navigation via Open-Set Semantic Memory and Behavior Adaptation Seungchan Kim, Omar Alama, Dmytro Kurdydyk, John Keller, Nikhil Keetha, Wenshan Wang, Yonatan Bisk, Sebastian Scherer Abstract Aerial outdoor semantic navigation requires robots to explore large, unstructured environments to locate target objects. Recent advances in semantic navigation have demonstrated open-set object-goal navigation in indoor settings, but these methods remain limited by constrained spatial ranges and structured layouts, making them unsuitable for long-range outdoor search. While outdoor semantic navigation approaches exist, they either rely on reactive policies based on current observations, which tend to produce short-sighted behaviors, or precompute scene graphs offline for navigation, limiting adaptability to online deployment. We present RAVEN , a 3D memory-based, behavior tree framework for aerial semantic navigation in unstructured outdoor environments. It (1) uses a spatially consistent semantic voxel-ray map as persistent memory, enabling long-horizon planning and avoiding purely reactive behaviors, (2) combines short-range voxel search and long-range ray search to scale to large environments, (3) leverages a large vision-language model to suggest auxiliary cues, mitigating sparsity of outdoor targets. These components are coordinated by a behavior tree, which adaptively switches behaviors for robust operation. We evaluate RAVEN in 10 photorealistic outdoor simulation environments over 100 semantic tasks, encompassing single-object search, multi-class, multi-instance navigation and sequential task changes. Results show RAVEN outperforms baselines by 85.25 % 85.25\% in simulation and demonstrate its real-world applicability through deployment on an aerial robot in outdoor field tests. Website: https://raven-semantic.github.io/ I INTRODUCTION Aerial robots are increasingly employed for autonomous exploration and search in outdoor environments. Prior research has largely focused on geometry-driven strategies, including frontier-based exploration [ 1 ] and volumetric information gain maximization [ 2 ] . While these methods efficiently cover the spaces, they rely on geometric representations–such as occupancy grids or volumetric maps–and cannot perform semantic reasoning, for example, locating a yellow car in urban areas or identifying machinery scattered across a construction site. Semantic navigation has been extensively studied in indoor environments [ 3 , 4 , 5 ] , where robots are tasked with reaching object goals or following language instructions. With the advent of foundation models, recent approaches have leveraged these models for open-set object navigation [ 6 , 7 , 8 ] . However, such methods remain largely confined to bounded 2D indoor settings [ 3 , 4 , 6 , 5 , 7 , 8 ] , where the spatial horizon is limited, the robot’s range of possible movements is constrained, and the structured environment provides cues for locating objects. By contrast, outdoor semantic navigation poses two major challenges compared to indoor settings. First, the much larger spatial extent of outdoor scenes requires long-range search strategies; second, the sparse distribution of target objects, combined with the absence of a structured hierarchy (e.g., building \rightarrow floor \rightarrow room \rightarrow object) demands careful strategic planning. Some prior methods [ 9 , 10 , 11 ] pursue map-free navigation with reactive policies that leverage short-term observation histories, which often generate short-sighted behaviors. Other works attempt semantic navigation in outdoor scenes using graph representations; however, they construct these graphs offline, which limits their online applicability [ 12 , 13 ] . We propose RAVEN , a R esilient A erial V oxel-Ray Memory E mpowered N avigation, a novel framework for outdoor semantic search. (1) RAVEN builds upon recent advances in open-set semantic voxel and ray representations [ 14 ] , leveraging them as persistent internal memory . This open-set voxel-ray memory not only avoids purely reactive behaviors and enables long-horizon planning, but also supports multi-class search and accommodates dynamic task switching, owing to its task-agnostic nature. (2) Beyond voxel-based search for close and reliable cues, RAVEN performs ray-based search to register and plan over long-range distance cues, thereby addressing the challenge of long-range reasoning in outdoor settings. (3) To overcome the semantic sparsity of outdoor scenes, it invokes a large vision language model (LVLM)-guided search when needed to incorporate auxiliary cues. These search behaviors are integrated through a behavior tree, which allows the system to adaptively select behaviors depending on situations and to maintain resiliency by switching strategies when one fails. We validate RAVEN in extensive simulation environments, and demonstrate its promising performance through real-robot deployment testing. In summary, our contributions are as follows: • We present a new aerial semantic navigation framework that leverages an open-set voxel–ray memory as an internal representation of the world and employs a behavior tree to robustly adapt across multiple search behaviors. • We address the challenges of long-range reasoning and sparse semantic cues via ray-based search and LVLM guidance within the behavior tree. Our method further supports multi-class search and on-the-fly task switching, an underexplored aspect of semantic navigation. • We validate the framework by demonstrating superior performance over existing baselines across diverse tasks in simulation and further showcase its real-world feasibility through deployment on a physical robot. II RELATED WORKS Mobile robot autonomy spans diverse tasks, from exploration to navigation, and building on these foundations, recent works have explored the semantic navigation paradigm [ 3 , 4 , 5 ] , where robots understand and reason about semantic information in the environment, such as object types, language, and context, to guide their navigation. With the rise of Vision Foundation Models (VFMs) [ 15 ] and LVLMs [ 16 ] , semantic navigation has seen tremendous growth [ 17 , 18 ] . Tasks in semantic navigation include vision-language navigation (VLN) [ 19 , 17 , 20 ] , where a robot executes a vision-grounded language instruction to follow paths; object-goal navigation (OGN) [ 3 , 6 , 21 ] , where a robot locates and navigates toward objects based on their categories or descriptions; embodied question answering (EQA) [ 18 ] , where a robot actively explores to answer questions about the environment. In this work, we focus on object-goal navigation. While prior studies have primarily focused on indoor, ground robots, and single-object scenarios, we aim to explore OGN in outdoor settings, with aerial robots, and multi-class, multi-object scenarios–an important yet widely underexplored domain. II-A Classical Exploration and Search Early work on robotic exploration began with frontier-based methods using 2D occupancy grids [ 22 ] . Extensions to 3D aerial scenarios include frontier-based fast exploration [ 1 ] and sampling-based approaches driven by information gain [ 2 ] . While these methods have shown strong performance in real-world deployments, their primary focus is on mapping and covering unknown areas, often neglecting semantic information. Recent efforts [ 23 ] employ aerial robots to explore regions and attempt to detect target objects. However, they treat object discovery merely as a byproduct of map coverage rather than performing object-goal-driven planning. In contrast, RAVEN leverages open-set semantic voxel-ray memory and LVLM to guide planning, while retaining the robustness of traditional methods when semantic guidance is insufficient. II-B Map-free Semantic Navigation Building on the success of vision language models (VLMs) and multi-modal LLMs, recent works explored zero-shot semantic navigation that uses semantic priors to interpret observations and directly generate control commands. Early approaches relied on contrastive VLMs such as CLIP [ 15 ] to sequentially guide the robot [ 7 ] . Later works leverage LLMs’ reasoning capabilities [ 24