Safe Planning in Unknown Environments using Conformalized Semantic Maps

David Smith Sundarsingh, Yifei Li, Tianji Tang, George J. Pappas, Nikolay Atanasov, Yiannis Kantaros · 2025

Read Safe Planning in Unknown Environments using Conformalized Semantic Maps by David Smith Sundarsingh, Yifei Li, Tianji Tang, George J. Pappas, Nikolay Atanasov, Yiannis Kantaros (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-25124.txt.

Introduction

David Smith Sundarsingh and co-authors present a 2025 robot planner that stays safe in unknown places despite fuzzy perception. A robot must reach target spots while keeping set distances from labeled objects. Sensor readings are unsure, so maps built on the fly carry error. Older planners either ignore that error or demand a known sensor model.

Their planner measures map uncertainty with conformal methods. It then plans paths that meet a user-set chance of success. No knowledge of sensor noise is needed. Semantic maps grow as the robot senses its surroundings. The method is the first of its kind for such reach-avoid tasks. It suits robots in homes, hospitals, and disaster zones.

Catalog record

CREATORDavid Smith Sundarsingh, Yifei Li, Tianji Tang, George J. Pappas, Nikolay Atanasov, Yiannis Kantaros
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE45 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2509-25124

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Safe Planning in Unknown Environments using Conformalized Semantic Maps David Smith Sundarsingh, Yifei Li, Tianji Tang, George J. Pappas, Nikolay Atanasov, Yiannis Kantaros Abstract This paper addresses semantic planning problems in unknown environments under perceptual uncertainty. The environment contains multiple unknown semantically labeled regions or objects, and the robot must reach desired locations while maintaining class-dependent distances from them. We aim to compute robot paths that complete such semantic reach-avoid tasks with user-defined probability despite uncertain perception. Existing planning algorithms either ignore perceptual uncertainty—thus lacking correctness guarantees—or assume known sensor models and noise characteristics. In contrast, we present the first planner for semantic reach-avoid tasks that achieves user-specified mission completion rates without requiring any knowledge of sensor models or noise. This is enabled by quantifying uncertainty in semantic maps—constructed on-the-fly from perceptual measurements—using conformal prediction in a model- and distribution-free manner. We validate our approach and the theoretical mission completion rates through extensive experiments, showing that it consistently outperforms baselines in mission success rates. I Introduction Autonomous robot navigation has received considerable research attention, leading to the development of numerous planners capable of designing trajectories that reach known goal regions while maintaining safe distances from obstacles [ 1 , 2 , 3 ] . Recent advances in AI-enabled robot vision and semantic mapping have created new opportunities to move beyond these well-studied geometric planning approaches toward semantic planning, where task completion requires reasoning not only about the geometric structure of the environment (e.g., obstacle locations) but also its semantic structure (e.g., object categories or terrain types) [ 4 , 5 ] . This paper focuses on semantic planning problems in environments populated with multiple objects and regions whose locations and semantic categories are initially unknown. The robot’s goal is to reach desired destinations while maintaining category-dependent safety distances from these semantic regions and objects. Recent planning algorithms tackle such tasks by leveraging AI-enabled perception systems (e.g., segmentation and object recognition algorithms) to collect geometric and semantic observations [ 6 , 7 , 8 , 9 , 10 , 11 , 12 , 13 , 14 , 15 , 16 , 17 ] . These approaches typically either (i) employ deep learning methods to directly map perceptual inputs to robot actions (e.g., [ 15 , 14 , 17 ] ) or (ii) leverage semantic SLAM techniques [ 18 , 19 , 20 ] to build semantic maps that are used by downstream planners (e.g., [ 6 , 7 , 21 ] ). Despite impressive empirical performance, in these architectures the planning layer generally treats outputs from the perception or mapping modules as ground truth, ignoring their inherent uncertainty. As a result, these approaches lack mission completion guarantees. Attempts to model perceptual uncertainty within semantic and geometric planning have been investigated in [ 6 , 7 , 22 ] and [ 23 , 24 , 25 , 26 , 27 ] , respectively. However, these methods often depend on unreliable proxies such as SLAM-derived confidence scores, or assume known sensor models and Gaussian noise—assumptions that may not hold in practice. Specifically: (i) SLAM-based uncertainty metrics often stem from fusing uncalibrated confidence scores from perception systems [ 28 ] ; (ii) sensor models are often nonlinear and poorly characterized in real environments; and (iii) noise is often non-Gaussian or multi-modal. Fig. 1: Illustration of the proposed planning framework for semantic reach-avoid tasks \psi . Given categorical and range measurements, a semantic map is constructed [ 19 ] . To account for map uncertainty caused by perceptual imperfections, conformal prediction is employed to generate sets of maps that contain the ground-truth environment with a user-defined probability. These sets are then used by a planner to generate uncertainty-aware paths that complete the assigned tasks with a user-defined probability 1-\alpha . This paper addresses these limitations by developing a semantic planner for unknown environments that completes semantic reach-avoid tasks with a user-defined probability under perceptual uncertainty, while remaining agnostic to sensor models and noise characteristics. Robots are equipped with imperfect perception systems (e.g., cameras with semantic segmentation) that provide geometric and semantic observations. Our framework consists of three main components; see Fig. 1 . First, we leverage existing semantic mapping algorithms that capture dense semantic and geometric environmental information and generate a multi-class semantic map of the environment [ 19 ] . Second, we quantify uncertainty of the generated semantic maps using Conformal Prediction (CP), a statistical, model- and distribution-free tool [ 29 ] . This produces a set of maps guaranteed to contain the ground truth with a user-defined probability. Third, we develop a semantic planner that balances exploitation and exploration. In exploitation mode, the planner selects the most conservative map from the CP set and generates paths using any search- or sampling-based planner [ 1 , 2 , 3 ] . When uncertainty is too high for a feasible solution to exist, the planner switches to exploration mode to reduce uncertainty (modeled by the size of the CP set) until a feasible path can be found. We show both theoretically and empirically that our method achieves desired mission completion rates. We also conduct experiments highlighting the exploitation–exploration trade-offs as the required success probability increases. Our comparative experiments also demonstrate that our approach outperforms baselines that either treat mapping outputs as ground truth or rely on uncertainty metrics provided by the mapping algorithm. Related Work: (i) Perception-based Planning: As discussed earlier, numerous planners for unknown environments have been proposed. However, these either neglect perceptual/mapping uncertainty and, thus, lack correctness guarantees [ 8 , 9 , 10 , 11 , 13 , 14 , 12 , 15 , 16 , 17 ] , or assume known sensor models and noise characteristics to design probabilistically correct paths [ 6 , 7 , 23 , 24 , 25 , 26 , 27 ] . In contrast, our planner achieves user-specified mission completion rates without assuming any knowledge of sensor models or noise. The closest work to ours is [ 30 ] , which employs CP to quantify the uncertainty of object detectors and design probabilistically correct paths. However, [ 30 ] focuses solely on geometric planning and cannot straightforwardly handle combined geometric and semantic uncertainty. (ii) CP for Verified Autonomy: CP has recently been applied to quantify the uncertainty of various AI-enabled components to ensure verified autonomy; see the survey in [ 31 ] . Applications include trajectory prediction models [ 32 , 33 ] , large language models [ 34 , 35 ] , decision-making algorithms [ 36 ] , image classifiers [ 37 ] , diffusion dynamic models [ 38 ] , and 3D human pose estimators [ 39 ] . Inspired by these works, we propose the first use of CP to quantify semantic map uncertainty, thereby enabling probabilistically correct semantic planning. Contributions : The contributions of this paper can be summarized as follows. First, we introduce the first semantic planner for unknown environments that achieves user-defined mission completion rates under perceptual uncertainty, while remaining agnostic to sensor models and noise characteristics. Second, we present the first application of CP to quantify the uncertainty of semantic maps generated by existing mapping algorithms, also in a model-agnostic manner. Third, we present experiments that validate the theoretical

Read the complete text