BeliefMapNav: 3D Voxel-Based Belief Map for Zero-Shot Object Navigation

Zibo Zhou, Yue Hu, Lingkai Zhang, Zonglin Li, Siheng Chen · 2025

Read BeliefMapNav: 3D Voxel-Based Belief Map for Zero-Shot Object Navigation by Zibo Zhou, Yue Hu, Lingkai Zhang, Zonglin Li, Siheng Chen (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-06487.txt.

Introduction

Zibo Zhou, Siheng Chen, and three co-authors help robots find objects in new places with no training. Their paper, from twenty twenty-five, presents BeliefMapNav. The robot gets a plain language target, like a chair or a mug. Big language and vision models can guess where such things live, but they plan greedily and lose the big picture.

Their system builds a 3D block map of belief over the space. Each block holds a chance that the target sits there. Language priors and live sightings update the map as the robot moves. Search then follows the full map, not the latest guess. The method grounds word reasoning in real space.

Catalog record

CREATORZibo Zhou, Yue Hu, Lingkai Zhang, Zonglin Li, Siheng Chen
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE53 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2506-06487

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

BeliefMapNav: 3D Voxel-Based Belief Map for Zero-Shot Object Navigation Zibo Zhou, Yue Hu, Lingkai Zhang, Zonglin Li, Siheng Chen Abstract Zero-shot object navigation (ZSON) allows robots to find target objects in unfamiliar environments using natural language instructions, without relying on pre-built maps or task-specific training. Recent general-purpose models, such as large language models (LLMs) and vision-language models (VLMs), equip agents with semantic reasoning abilities to estimate target object locations in a zero-shot manner. However, these models often greedily select the next goal without maintaining a global understanding of the environment and are fundamentally limited in the spatial reasoning necessary for effective navigation. To overcome these limitations, we propose a novel 3D voxel-based belief map that estimates the target’s prior presence distribution within a voxelized 3D space. This approach enables agents to integrate semantic priors from LLMs and visual embeddings with hierarchical spatial structure, alongside real-time observations, to build a comprehensive 3D global posterior belief of the target’s location. Building on this 3D voxel map, we introduce BeliefMapNav, an efficient navigation system with two key advantages: i) grounding LLM semantic reasoning within the 3D hierarchical semantics voxel space for precise target position estimation, and ii) integrating sequential path planning to enable efficient global navigation decisions. Experiments on HM3D, MP3D, and HSSD benchmarks show that BeliefMapNav achieves state-of-the-art (SOTA) Success Rate (SR) and Success weighted by Path Length (SPL), with a notable 46.4% SPL improvement over the previous best SR method, validating its effectiveness and efficiency. The code is available on BeliefMapNav. 1 Introduction Zero-shot object navigation(ZSON) enables robots to locate targets in novel environments through natural language instructions (e.g., "find the red sofa"), eliminating reliance on pre-mapped scenes or object-specific training ( chaplot2020learning ; chaplot2020object ; zhou2023esc ; zhou2024navgpt ; yao2023react ) . In domestic settings, ZSON supports assistive tasks such as retrieving user-specified objects ( liu2024ok ) . In industrial inspection, ZSON enables autonomous localization of malfunctioning components (e.g., detecting a leaking pipe) within complex facilities. In warehouse operations, ZSON enhances robotic picking and inventory management by allowing flexible retrieval of objects without pre-built maps. These real-world applications highlight the necessity of robust zero-shot navigation for scalable, adaptable robot deployment across diverse domains. Figure 1 : The search process: BeliefMapNav plans frontier paths by minimizing the expected search distance based on the 3D voxel-based belief map, ensuring efficient and stable exploration. To enable ZSON, prior works have progressed along two main directions. The first constructs bird’s-eye view (BEV) value maps ( yokoyama2024vlfm ; huang2024gamap ; long2024instructnav ) , by leveraging pixel-level semantic cues, these methods provide dense estimations of target locations in the BEV space. While dense BEV representations provide object position estimation at a fine-grained level, the actual numerical distinctions between positions remain ambiguous with limited discriminative semantics. This insufficient differentiation in positional values, combined with the lack of high-level semantic reasoning, ultimately leads to compromised accuracy in the target location prediction. More recently, a second approach has emerged that employs large language models (LLMs) or vision-language models (VLMs) to reason about the target locations ( goetting2024end ; cai2024bridging ; yin2024sg ; gadre2023cows ; yu2023l3mvn ) . However, both LLMs and VLMs face limitations in spatial understanding and reasoning ( yang2024thinking ) , which significantly affect target location prediction accuracy. Additionally, VLMs are limited by their inability to effectively extract the most relevant and fine-grained semantic information from image observations, as well as by the lack of spatial information, resulting in imprecise target position predictions ( goetting2024end ; cai2024bridging ; yin2024sg ; gadre2023cows ; yu2023l3mvn ) . For LLMs, while leveraging spatial information, reliance on semantic content from environment language descriptions leads to significant loss of information and reduced prediction precision ( chen2023not ; yin2024sg ) . Consequently, LLMs and VLMs both struggle with reliable and accurate target location inference. Furthermore, existing methods generally rely on greedy navigation strategies ( yin2024sg ; huang2024gamap ) , which cause frequent back-and-forth movements, significantly hindering search efficiency. Together, in existing works, the lack of semantic cues and spatial reasoning leads to inaccurate and imprecise target object position estimation. Meanwhile, the absence of efficiency optimization in the search behavior hinders robust searching for diverse, open-set targets in unbounded real-world 3D spaces. To enable more precise and accurate predictions of the target object’s location within 3D space, we propose a novel 3D voxel-based belief map that considers rich hierarchical spatial semantic cues and LLM-generated target-adaptive semantic cues to reason about prior belief of target presence in dense 3D voxel space. This structured representation enables spatially fine-grained estimation across the 3D voxel space, facilitating more precise and generalizable localization of target objects in complex, unbounded environments. Moreover, this fine-grained and accurate representation enables efficient guidance for high-degree-of-freedom mobile agents, facilitating precise, task-relevant, language-driven search within localized areas and enhancing the robustness of manipulation tasks. To further enhance search efficiency, we introduce BeliefMapNav, an efficient zero-shot object navigation system based on path sequence optimization over the belief map. The system is composed of three tightly integrated modules and builds upon the belief map framework to enable efficient, goal-directed exploration. 1)The 3D voxel-based belief mapping module encodes prior beliefs of object presence in 3D space by integrating hierarchical spatial semantics with commonsense knowledge from an LLM. 2)The frontier observation belief estimation module combines the belief map with a visibility map, which encodes real-time observation feedback likelihood, to produce posterior beliefs of the target object’s position. Then, the module estimates the posterior observation belief of detecting the target in each frontier’s field of view (FOV). 3) The observation belief-based planning module selects the next navigation goal by minimizing the expected distance cost based on the posterior observation beliefs to find the target. By explicitly modeling uncertainty and updating spatial beliefs dynamically, BeliefMapNav enables more efficient, goal-directed exploration than methods using static priors or reactive policies. The contributions of our method are mainly summarized as follows: 1)We propose BeliefMapNav, an efficient zero-shot object navigation system that accurately predicts target location through fine-grained belief estimation in a 3D voxel-based belief map and real-time feedback, enabling belief-driven sequential planning for efficient navigation. 2) At the core of our system is a 3D voxel-based belief map that integrates hierarchical spatial-visual features with LLM-derived commonsense, enabling accurate and fine-grained prior estimation of the target’s location. Based on the prior estimation, we design a planner that optimizes navigation by minimizing the expected path distance cost for efficient navigation. 3)The proposed system achieves SOTA performance on the HM3D ( ramakrishnan2021habitat ) , MP3D ( chang2017matterport3d ) and HSSD ( khanna2024habitat ) benchmarks, surpassing all the previous zero-shot methods by 5.86% in SR and 46.4% in SPL on HM3D, 27.8% in SR and 28.9% in SPL on HSSD, thereby demonstrating the overall effectiveness and efficiency of our approach. 2 Related Works Object Navigation Object navigation refers to the task of guiding a robot to search given target objects in an unknown environment. It can be divided into two categories: i) training-required methods, such as reinforcement learning (RL) and imitation learning, which require extensive training on task-specific data ( wijmans2019dd ; mousavian2019visual ; yang2018visual ; majumdar2022zson ; maksymets2021thda ) , and ii) zero-shot methods, which leverage pre-trained models, such as VLMs or LLMs, to perform navigation without

Read the complete text