Geometry-Aligned LLM Fine-Tuning for Sequential Narrow-Opening Planning
Al Jaber Mahmud, Xuan Wang · 2026
Read Geometry-Aligned LLM Fine-Tuning for Sequential Narrow-Opening Planning by Al Jaber Mahmud, Xuan Wang (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-16028.txt.
Introduction
Mahmud and Wang teach a language model to plan through tight sequential openings. Early pose choices cage what later openings allow a rigid body. Their fine-tuning emits fixed-length waypoint lists in machine-readable form. A two-level pipeline shapes the model in stages. Failure-driven LoRA training on human demos comes first. Group policy optimization then refines the same adapters. A model planner densifies each path for geometry scoring. Top success rates hold in known and novel scenes. Exit poses show foresight for the next opening.
Catalog record
| CREATOR | Al Jaber Mahmud, Xuan Wang |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 46033 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.16028
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Geometry-Aligned LLM Fine-Tuning for Sequential Narrow-Opening Planning Al Jaber Mahmud, Xuan Wang Abstract We study rigid-body motion planning through multiple sequential narrow openings, which requires long-horizon geometric reasoning because the configuration used to traverse an early opening constrains the set of reachable configurations for subsequent ones. To achieve this, we propose a geometry-aligned large language model (LLM) fine-tuning framework that generates fixed-length, machine-readable waypoint sequences that are both geometrically feasible and coordinated across openings. Our approach uses a bi-level training pipeline. First, we perform failure-driven LoRA supervised fine-tuning (SFT) on human demonstrations, which incorporates structured failure feedback to teach the model common failure modes and enforce the output format. Second, we refine the same LoRA adapters using Group Relative Policy Optimization (GRPO) with geometric verification: each sampled waypoint sequence is densified by a model-based planner and scored with a deterministic geometry-derived reward to achieve continuous-motion feasibility. To validate the effectiveness of our proposed method, we provide both quantitative and qualitative results from simulations. Our method achieves the highest success rate in both in-distribution and out-of-distribution environments and qualitatively exhibits long-horizon geometric reasoning by selecting exit poses that facilitate entry into subsequent openings. I Introduction A common problem in manipulation, logistics, and co-transportation [ 1 ] is enabling rigid-body navigation through cluttered environments, which often requires passing through multiple narrow openings (e.g., doors or constrained paths between obstacles) in sequence. Unlike single-passage planning, where feasibility can be determined locally, navigating multiple openings requires sequential geometric reasoning: the configuration used to traverse an early opening constrains the set of reachable configurations for subsequent ones. For example, when carrying a large object through a doorway or co-transporting it in a cluttered space, the object must enter the first opening in a configuration that not only avoids immediate collision but also preserves sufficient clearance and maneuverability to rotate and align for the next passage. This observation implies that an effective solver must perform long-horizon geometric reasoning across sequential openings, selecting early configurations with future constraints explicitly taken into account. Fig. 1: Geometry-aligned LLM for Sequential Narrow-Opening Planning and a Motivating Example. Motivating Example: As illustrated in Fig. 1 , the object pose used to pass through the first opening must also support entry into the second one. Here, even though multiple configurations allow traversal of the first opening (lower left), only a limited subset remains feasible for the second opening (upper right), where the geometric constraints are more restrictive. Since the corridor is too narrow to permit reorientation, the choice made at the first opening largely determines feasibility at the second. Therefore, successful planning requires coordinated reasoning over the full sequence of passages, rather than independent local feasibility checks. The requirements illustrated by the above example are not fully satisfied by existing model-based or learning-based approaches. Model-based planners can enforce geometric correctness for collision avoidance, but their performance in multiple sequential narrow openings is often sensitive to implementation choices such as workspace discretization, sampling density, heuristics, and cost tuning [ 2 , 3 ] . This makes it harder to find globally feasible solutions within a reasonable time budget. On the other hand, if we decompose the task into intermediate subgoals and plan one opening at a time, it can improve efficiency [ 4 , 5 ] . However, it may face issues when composing trajectories: locally feasible exit poses from one opening may not admit a feasible further trajectory in subsequent openings. Learning-based planners can generate waypoints faster at test time [ 6 ] . However, they usually do not provide geometric guarantees for collision-free continuous motion between waypoints. Their performance may drop when the obstacle layout or rigid body shape differs from the training distribution [ 7 ] . To address the limitations of existing model-based and learning-based planners in sequential narrow-opening navigation, this paper introduces a geometry-aligned LLM fine-tuning framework for motion planning. The framework integrates long-horizon reasoning with structured geometric feedback and optimization-based evaluation. An overview of the framework is given in Fig. 1 . The main contributions are threefold: • We leverage the sequential reasoning capability of large language models and fine-tune an LLM to generate a fixed-length, machine-readable \mathrm{SE}(2) waypoint sequence structured into per-opening waypoint blocks. This structured output interface enables long-horizon coordination across sequential openings by explicitly selecting exit poses that facilitate feasible entry into subsequent openings. • We introduce a failure-driven training loop in which a deterministic geometric verifier evaluates the densified trajectory (from LLM output) and returns the first failing waypoint index together with the violated constraint type (boundary, collision, swept collision, or step-size). This structured feedback is incorporated into Low Rank Adaptation (LoRA) based supervised fine-tuning (SFT), allowing the model to iteratively reduce recurring geometric errors rather than merely imitating demonstrations. • We further refine the same LoRA adapters using Group Relative Policy Optimization (GRPO) with geometric verifier, where each sampled waypoint sequence is densified and evaluated by the geometric verifier to produce a deterministic reward. This reward directly reflects continuous-motion feasibility under geometric constraints, thereby optimizing the model beyond waypoint-level validity and aligning generation with full trajectory correctness. We demonstrate the effectiveness of the proposed approach through experiments and comparison with baseline methods. II Related Work II-A Model-Based Motion Planning In model-based motion planning, the planner searches for collision-free paths in configuration space (C-space), where feasibility can be evaluated with deterministic geometric checks. For example, graph-based planners such as A^{*} and D^{*} (or D^{*}\text{-Lite} ) perform shortest-path search over discretized grids [ 8 , 9 ] . Sampling-based planners such as Probabilistic Roadmaps (PRM) and Rapidly-Exploring Random Trees (RRT/RRT-Connect) explore by randomly sampling states and connecting nearby samples to approximate connectivity in free space [ 10 , 11 ] . Both graph-based and sampling-based methods face challenges in narrow-opening problems, where feasible motions lie in thin regions of C-space and often require precise orientations to pass through an opening [ 10 , 12 ] . Thus, uniform path search or random sampling may require extensive tuning of key parameters such as grid resolution, sampling budget, connection radius, step size, and the difficulty increases with multiple sequential narrow openings due to geometric constraints and non-convexity. As a result, planners may fail to find a globally feasible path within a reasonable time budget. One way to overcome this is to decompose the whole task into a sequence of subgoals (each opening as an independent goal) and plan one opening at a time [ 4 , 5 ] . While this improves efficiency, the subpaths may fail to compose into a globally feasible trajectory since an exit pose that is feasible for the current opening, but leaves insufficient space to reorient and align with the next opening. II-B Learning-Based Planning Learning-based planners use data-driven models to predict trajectories or to guide model-based planners by learning where to search in the C-space. This can reduce the planning effort and computation at test time. For example, MPNet-style architectures map obstacle information and start-goal poses to continuous paths [ 6 ] . A network can be trained to propose sampling distributions that improve sampling-based planning in challenging regions [ 7 ] . Other approaches learn cost maps that are then optimized by motion planners such as CHOMP, TrajOpt, or