Follow Everything: Goal-Aware Adaptation and Graph-Based Planning Towards Arbitrary Leader Following
Qianyi Zhang, Shijian Ma, Boyi Liu, Jingtai Liu, Jianhao Jiao, Dimitrios Kanoulas , · 2025
Read Follow Everything: Goal-Aware Adaptation and Graph-Based Planning Towards Arbitrary Leader Following by Qianyi Zhang, Shijian Ma, Boyi Liu, Jingtai Liu, Jianhao Jiao, Dimitrios Kanoulas , (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-19399.txt.
Introduction
Qianyi Zhang and four co-authors build a robot leader-following system in twenty twenty-five. A robot that can follow a leader of any form could fit better into human settings. Older methods work only for set types of leaders. They also lose the leader when it leaves the robot's view for a short time.
Their framework swaps detection models for a segmentation model, so the leader can be anything. A buffer stores leader features at many distances to keep recognition strong. A goal-aware mechanism shifts planning states based on whether the leader is visible and how it moves. A graph-based planner then draws candidate paths for each state. Readers get a follower built to cope with real, messy scenes.
Catalog record
| CREATOR | Qianyi Zhang, Shijian Ma, Boyi Liu, Jingtai Liu, Jianhao Jiao, Dimitrios Kanoulas , |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 39 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2504-19399
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Follow Everything: Goal-Aware Adaptation and Graph-Based Planning Towards Arbitrary Leader Following Qianyi Zhang, Shijian Ma, Boyi Liu, Jingtai Liu, Jianhao Jiao, Dimitrios Kanoulas , Abstract Enabling robots to robustly follow leaders supports tasks such as carrying supplies or guiding customers. While existing methods often fail to generalize to arbitrary leaders, such as humans or other robots, and struggle when the leader temporarily leaves the robot’s field of view, this work presents a unified framework to address both challenges. First, a segmentation model replaces traditional category-specific detection models, allowing the leader to be of any shape or type. To improve robustness, a distance frame buffer is designed to store high-confidence leader embeddings across distance intervals, accounting for the unique characteristics of leader-following tasks. Second, a goal-aware adaptation mechanism is designed to govern robot planning states based on the leader’s visibility and motion, complemented by a graph-based planner that generates candidate trajectories for each state, ensuring efficient following with obstacle avoidance. Simulations and real-world experiments with a legged robot follower and diverse leaders (e.g., humans, UAVs, legged robots, and stop signs) in indoor and outdoor settings demonstrate an improved follow success rate of 75.1%, a reduced visual loss of 13.1%, a fewer collisions of 65.1%, and a shorter average distance of 0.4 m. Visit the website for the video and code. I Introduction Recent advances in visual perception, robot navigation, and large models have enabled robots to follow leaders with increased safety and intelligence [ 1 , 2 , 3 ] . Serving as powerful assistants, robots can follow explorers to carry more supplies, guide customers and introduce products in shopping centers, help police patrolling, etc. [ 4 , 5 , 6 ] . Although numerous works have been proposed to improve leader following stability [ 7 , 8 , 9 ] or robot movement smoothness [ 10 , 11 , 12 , 13 , 14 , 15 ] , all overlook a unique challenge in the leader-following task: the leader may temporarily leave the robot’s field of view (FOV), as shown in Fig. 1 (a), which causes existing methods to fail in finding a trajectory without knowing the leader’s position or in not recognizing the leaders when they reappear. This paper breaks down this challenge from both the perception and planning perspectives and proposes the distance frame buffer, goal-aware adaptation, and graph-based planner to address it. Fig. 1: Illustration of the challenges in leader-following tasks. (a) When the leader steps over a box that the follower must detour around, the leader leaves the follower’s field of view. (b) The follower is expected to follow the leader via a shorter trajectory, not by replicating the leader’s past path. The challenge on the perception side concerns whether the leader can be identified as the same leader when they leave and then reappear in the follower’s FOV. One specific approach for leader-following tasks is to have the leader carry a beacon, such as infrared antennas [ 16 ] or wave emitters [ 17 , 18 ] , but requiring the leader to carry additional equipment is often impractical and not generalizable. Therefore, another widely used solution is to leverage detection models based on cameras [ 7 ] , LiDAR [ 8 ] , multimodal sensors [ 9 ] , sonar [ 19 ] , and so on. However, since their training labels correspond to object categories rather than object features, when the leader (typically a pedestrian) remains in the follower’s FOV, its ID remains stable due to the help of the Kalman Filter [ 20 ] . However, when the leader leaves and reappears, a new ID will be assigned to it. This leads to the expected leader having the original ID permanently lost. Fig. 2: Illustration of the proposed follow everything framework. Given a leader prompt and an RGB image, the leader mask is segmented with the aid of both a memory buffer and a distance frame buffer. The segmented leader is then filtered from raw laser point clouds to produce a leader point set, which is passed to the goal-aware adaptation to dynamically adjust parameters and provide goal sets and constraints according to the follower-leader interaction. Given the costmap, goal sets, and constraints, the graph-based planner searches, optimizes, and selects an optimal trajectory for the follower to execute. An alternative solution is segmentation models [ 21 ] , which use features as labels. After the initial feature is provided, the model continuously tracks and bootstraps-updates the leader feature, allowing the leader to be reidentified when it reappears [ 10 ] . Not limited to categories, this type of model also enables anything with an arbitrary shape to be the leader, such as humans, vehicles, irregular objects, or even other robots. However, a limitation of existing segmentation datasets is that they often involve a fixed camera position (follower) with the leader’s movement, location and size constantly changing in the video, where bootstrapping is beneficial. However, in the leader-follower task, the follower is also in motion, making it easy for the leader to leave the follower’s FOV, as illustrated in Fig. 1 (a). Just before leaving, the leader often appears on the edge of the robot’s FOV, resulting in partial and low-quality features. Despite their poor quality, these features can still dominate the bootstrapping process, since the older features that capture the full-body shape of the leader differ significantly from the current appearance of the leader [ 10 ] . This can mislead the robot to bootstrap in the wrong direction. As a result, when the leader reappears later, the robot may not be able to retrace the leader accurately, even if the full leader is in front of it. To address this issue, this work introduces the distance frame buffer to SAM2 (see Sec. II-B ), which divides the distance ranges between the follower and leader into equal intervals and maintains the historical leader feature with the highest confidence in each distance interval, greatly improving the ability to retrace the leader after it reappears. The challenge on the planning side concerns obstacle avoidance and robust motion planning when the relationship between the leader and the follower is dynamic. While most existing approaches simply treat the leader’s position as a target and use PID [ 10 ] , MPC [ 11 ] , sampled trajectories [ 12 ] , optimized trajectories [ 13 ] , or learning-based methods [ 15 ] to plan trajectories for the follower (robot), they often lack robustness and fail to consider the various interactive states that arise from the leader-follower relationship. For example, (a) when the leader is within the follower’s FOV and close by, the main goal of the follower is to avoid obstacles while following, a scenario that frequently occurs when a legged robot follows a UAV leader, as shown in Fig. 6 , where the UAV can fly over short obstacles that the ground robot must detour around. (b) When the leader is within the follower’s FOV but at a greater distance, the main goal of the follower is to quickly approach the leader, which requires the robot to maintain high speed and choose optimal time trajectories to avoid increasing the risk of losing the leader