SmallPlan: Leverage Small Language Models for Sequential Path Planning with LLM-Guided Distillation and Reinforcement Learning from Simulation Feedback
Quang P. M. Pham, Khoi T. N. Nguyen, Nhi H. Doan, Cuong A. Pham, Qinbo Sun, Weimin Qi, Kentaro Inui, Dezhen Song · 2025
Read SmallPlan: Leverage Small Language Models for Sequential Path Planning with LLM-Guided Distillation and Reinforcement Learning from Simulation Feedback by Quang P. M. Pham, Khoi T. N. Nguyen, Nhi H. Doan, Cuong A. Pham, Qinbo Sun, Weimin Qi, Kentaro Inui, Dezhen Song (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-00831.txt.
Introduction
Big language models plan robot routes well but are too heavy to run on small devices at the edge. Quang Pham led a team that built SmallPlan, a way to shrink that skill into small language models. A big model acts as teacher: the small models learn from its guidance through rounds of training mixed with practice in simulation, where feedback rewards shorter, smarter routes.
The small models learn to output step-by-step actions across a simple graph that stands in for a full 3D scene. In tests they matched a top-tier big model on route planning, without its wild guesses or overfit answers. The result is planning that fits on modest hardware in real robots.
Catalog record
| CREATOR | Quang P. M. Pham, Khoi T. N. Nguyen, Nhi H. Doan, Cuong A. Pham, Qinbo Sun, Weimin Qi, Kentaro Inui, Dezhen Song |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 35 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-00831
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
SmallPlan: Leverage Small Language Models for Sequential Path Planning with LLM-Guided Distillation and Reinforcement Learning from Simulation Feedback Quang P. M. Pham, Khoi T. N. Nguyen, Nhi H. Doan, Cuong A. Pham, Qinbo Sun, Weimin Qi, Kentaro Inui, Dezhen Song Abstract Efficient path planning in robotics, particularly within large-scale, complex environments, remains a significant hurdle. While Large Language Models (LLMs) offer strong reasoning capabilities, their high computational cost and limited adaptability hinder real-time deployment on edge devices. We present SmallPlan - a novel framework leveraging LLMs as teacher models to train lightweight Small Language Models (SLMs) for high-level path planning tasks. In SmallPlan, the SLMs provide optimal action sequences to navigate across scene graphs that compactly represent full-scaled 3D scenes. The SLMs are trained in a simulation-powered, interleaved manner with LLM-guided supervised fine-tuning (SFT) and reinforcement learning (RL). This strategy not only enables SLMs to successfully complete navigation tasks but also makes them aware of important factors like distance travel, providing more efficient path planning. Through experiments, we demonstrate that the fine-tuned SLMs perform competitively with larger models like GPT-4o on sequential path planning, without suffering from hallucination and overfitting. SmallPlan is resource-efficient, making it well-suited for edge-device deployment and advancing practical autonomous robotics. Our source code is available here: https://github.com/quangpham2006/SmallPlan I Introduction Recent advances in LLMs have played a key role in pushing the boundaries of robotic fields, particularly in scene understanding and path planning, due to their capability in semantic reasoning and problem-solving. [ 1 , 2 , 3 ] works on foundational models that utilize LLMs for common-sense reasoning and task planning. Leveraging these works, [ 4 , 5 ] propose Vision-Language Models for long-horizon planning tasks, aiming to enable robots to perceive and traverse large-scale and complex environments. More recent studies [ 6 , 7 ] explore LLMs to generate long-horizon navigation plans from topological maps [ 8 , 9 ] or graph-based scene representations (scene graph) [ 10 , 11 , 12 , 13 ] . Compared to frame-by-frame reasoning with full-scale 3D representations, these approaches offer a lighter and compact alternative while retaining critical global semantic and spatial context. Fig. 1 : LLMs are powerful but often costly, SLMs are efficient but lack reasoning. We bridge the gap—training SLMs for sequential path planning leveraging LLMs, scene graph, SFT, and RL. However, these approaches still rely either on heavy-weight LLMs, which are computationally expensive, or on LLM-as-a-Service (GPT-4o, GPT-4) [ 6 , 14 ] , which incur high API costs. Such setups are impractical for edge deployment and raise concerns about carbon emissions [ 15 ] . In addition, using off-the-shelf LLMs for direct inference makes it difficult to control important factors such as the distance or cost of the planning sequences they generate, which are crucial for managing energy consumption on edge devices. Therefore, we propose SmallPlan , a distillation framework that uses LLMs as teacher models to train the SLMs for high-level path planning. Our framework enhances the outputs of SLMs by incorporating two key components. Firstly, the candidate SLM learns not only from the teacher LLM but also from real-time simulation feedback during training, achieving a balance between following a stronger model and adapting to surrounding environments. Secondly, we leverage RL and design a custom reward policy that incentivizes the SLM to control the travel distance and number of trials, making the inference less resource-consuming. Our key contributions, demonstrated in Fig. 1 , include: • We enhance SLMs for sequential high-level path planning by combining LLM-guided distillation fine-tuning and RL from simulation feedback. • We show that SLMs can achieve competitive performance with LLMs despite significantly lower resource requirements. SmallPlan enhances the SLMs’ capability by learning from superior LLMs. Moreover, with RL and our reward policy, SmallPlan controls important factors to make the planning more efficient, enhancing real-world deployability. • We study different training strategies that contribute to the outcome SLMs. We analyze the critical role of SFT distillation in stabilizing the SLMs’ outputs, and how the SLMs adapt to different reward penalties in RL. II Related Works Scene Understanding and Representation. A primary challenge in path planning is enabling robots to perceive and traverse large-scale and complex environments. VLMs have become a prevalent solution by leveraging foundation model reasoning for long-horizon planning tasks [ 4 , 5 ] . However, these approaches often operate on a per-frame basis, lacking global scene understanding. Such localized processing limits their performance on large-scale navigation tasks, as individual frames provide only partial and fragmented views of the surroundings. Therefore, recent efforts utilized topological maps [ 8 , 9 ] and scene graphs [ 10 , 11 , 12 , 13 ] to encode semantic and spatial relationships from raw sensor data (point clouds, videos, images). From these compact representations, LLMs can be applied for higher level tasks. MapGPT [ 7 ] integrates topological maps, vision models, and LLM prompt management, to enhance adaptive path planning. SayPlan [ 6 ] , MoMaLLM [ 14 ] employ scene graphs to generate long-horizon navigation plans with LLMs. Our approach follows this direction, using scene graph instead of full-scale 3D representation, due to its efficiency and effectiveness in representing semantic and spatial relationships. LLMs for Planning and Reasoning Tasks. Despite exhibiting impressive capability, LLMs sometimes struggle with reasoning tasks that are relatively simple for humans. To address this, [ 16 , 17 , 18 , 19 ] experiment the effect of different prompt templates in LLMs’ reasoning, in search of optimal prompting strategies. More recent research, on the other hand, highlights the adaptability of LLMs to human-defined rules [ 20 ] , suggesting promising applications in long-horizon planning tasks in robotics [ 21 , 14 , 22 , 23 ] . However, existing methods rely on pre-trained LLMs for direct reasoning without fine-tuning, leading to inefficient re-planning and inexecutable action sequences [ 6 ] . Additionally, they often require iterative planning cycles to produce executable plans, which is computationally intensive and cost inefficient for on-device inference. Our approach aims to replaces LLMs with lighter SLMs while retaining strong reasoning capability with knowledge distillation. Definition of SLMs. Categorizations of SLMs varies across contexts [ 24 ] . For example, [ 25 ] defines SLMs as models with fewer than 1 billion parameters, while [ 26 , 27 , 28 ] consider the term SLMs relative to their larger counterparts. For mobile devices, typically possessing around 6GB of memory, [ 29 ] suggests SLMs should have fewer than 10 billion parameters. NVIDIA [ 30 ] similarly considers models under 10 billion parameters as SLMs. In this work, we study the SLMs with at most 4 billion parameters and require less than 4GB of memory for inference. Distillation for SLMs. Multiple studies [ 31 , 32 , 33 , 34 , 35 ] have explored techniques for transferring knowledge from large models (LLMs) to smaller ones (SLMs). One common strategy is feedback-driven fine-tuning [ 36 ] . Other works [ 37 , 38 ] suggest that feedback from mistakes and environment can improve fine-tuning in robotics task planning. [ 39 ] proposes a SFT distillation framework for turning point robot navigation. To further explore the capability of SLMs, we propose a multi-objective