Generalizable Geometric Prior and Recurrent Spiking Feature Learning for Humanoid Robot Manipulation

Xuetao Li İD, Wenke Huang İD, Mang Ye İD, Jifeng Xuan İD · 2026

Read Generalizable Geometric Prior and Recurrent Spiking Feature Learning for Humanoid Robot Manipulation by Xuetao Li İD, Wenke Huang İD, Mang Ye İD, Jifeng Xuan İD (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-09031.txt.

Introduction

Xuetao Li and three co-authors work on human shaped robots that must grasp and move things. Such a robot needs high thought, to know what a task asks. It needs low skill too, to move arms with care. Two hard parts block it. It needs a true read of a 3D scene. It also must learn from few human demos, not vast sets.

The team adds a shape based prior that holds across scenes. Prior here means built in shape sense the robot can reuse. They add a spiking learn part that fires in bursts, like nerve cells. It aims to learn fast from few demos. A reader in robot work gets a plan for robots that can shift to new rooms and tasks.

Catalog record

CREATORXuetao Li İD, Wenke Huang İD, Mang Ye İD, Jifeng Xuan İD
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE81 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2601-09031

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Generalizable Geometric Prior and Recurrent Spiking Feature Learning for Humanoid Robot Manipulation Thanks: The authors are with the School of Computer Science, School of Robotics, Institute of Technological Sciences, Wuhan University, Wuhan, China 430072. E-mail:{xtli312, wenkehuang, yemang, jxuan, dubo, shengliu, miao.li}@whu.edu.cn, (Xuetao Li and Wenke Huang contributed equally, Jifeng Xuan and Miao Li are corresponding authors). A preliminary version of this work has appeared in Proceedings of the AAAI Conference on Artificial Intelligence (AAAI 2026) [ 56 ] . Xuetao Li İD, Wenke Huang İD, Mang Ye İD, Jifeng Xuan İD Abstract Humanoid robot manipulation is a crucial research area for executing diverse human-level tasks, involving high-level semantic reasoning and low-level action generation. However, precise scene understanding and sample-efficient learning from human demonstrations remain critical challenges, severely hindering the applicability and generalizability of existing frameworks. This paper presents a novel RGMP-S, Recurrent Geometric-prior Multimodal Policy with Spiking features, facilitating both high-level skill reasoning and data-efficient motion synthesis. To ground high-level reasoning in physical reality, we leverage lightweight 2D geometric inductive biases to enable precise 3D scene understanding within the vision-language model. Specifically, we construct a Long-horizon Geometric Prior Skill Selector that effectively aligns the semantic instructions with spatial constraints, ultimately achieving robust generalization in unseen environments. For the data efficiency issue in robotic action generation, we introduce a Recursive Adaptive Spiking Network. We parameterize robot-object interactions via recursive spiking for spatiotemporal consistency, fully distilling long-horizon dynamic features while mitigating the overfitting issue in sparse demonstration scenarios. Extensive experiments are conducted across the Maniskill simulation benchmark and three heterogeneous real-world robotic systems, encompassing a custom-developed humanoid, a desktop manipulator, and a commercial robotic platform. Empirical results substantiate the superiority of our method over state-of-the-art baselines and validate the efficacy of the proposed modules in diverse generalization scenarios. To facilitate reproducibility, the source code and video demonstrations are publicly available at https://github.com/xtli12/RGMP-S.git . I Introduction Humanoid robots represent a pivotal frontier in embodied intelligence, capable of merging high-level cognition with dexterous physical interaction [ 52 , 28 ] , largely propelled by the emergence of foundation models trained on massive-scale datasets [ 63 ] . However, in real-world deployment, robotic agents face inherently unstructured environments governed by complex physical laws, which differ significantly from the static internet data distributions used in pre-training [ 24 , 22 ] . Due to the infinite variability of 3D scenes and the exorbitant cost of collecting high-fidelity interaction data, the prevailing ”brute-force” scaling paradigm, which relies on exhaustive statistical memorization, becomes prohibitive and sample-inefficient for mastering dynamic manipulation tasks [ 1 ] . Driven by such real-world challenges, augmenting architectures with explicit geometric reasoning and spatial awareness has been increasingly explored, as it equips agents with structural inductive biases to decipher physical constraints effectively [ 47 ] . Fig. 1: Overview of our framework. By integrating geometric commonsense with spatiotemporal features, our RGMP-S encodes the robot-target spatial relationships for manipulation tasks. RGMP-S achieves a 19% performance improvement and exhibits 5\times greater data efficiency compared to the Diffusion Policy (DP) baseline. Vision-Language Models (VLMs), exemplified by PaLM-E and InstructBLIP, excel at parsing spatial semantic information from multimodal inputs to generate logical robotic task plans [ 12 , 11 , 30 ] . However, a critical gap remains in the execution phase: current architectures struggle to translate abstract instructions into contextually appropriate robotic skills [ 51 ] . The fundamental bottleneck lies in the 3D geometric grounding process, where VLMs, despite their proficiency in object identification, often fail to infer the physical interaction affordances dictated by specific geometries [ 2 ] . Specifically, regarding skill selection in open-world environments, this disconnect leads to a breakdown in differentiating manipulation primitives. Standard VLMs often confuse geometrically sensitive actions, such as grasping versus pinching, when encountering diverse object shapes [ 44 , 50 ] . For instance, a model trained to grasp standard Coke cans typically targets the cylindrical sidewall. When presented with a crushed Coke can, the VLM retains semantic recognition but fails to perceive the structural deformation. Consequently, the robot attempts to execute the original side-grasping primitive, which is no longer physically feasible for the flattened geometry, resulting in task failure, as illustrated in Fig. 1 . In fact, the flattened profile necessitates a pinching motion rather than the conventional side grasp. Such failures stem from a lack of explicit geometric reasoning . Existing approaches usually combine spatial indicators such as bounding boxes or contact points with high-level task descriptions [ 35 , 10 ] . However, these methods predominantly rely on coarse semantic embeddings. In dynamic settings, such dependence results in plans that maintain semantic validity yet remain physically inexecutable [ 42 ] . This deficiency emphasizes the critical necessity for systems capable of verifying skill feasibility through rigorous geometric analysis. This leads to a core research question: I) How can robots effectively exploit geometric reasoning to reliably select feasible manipulation skills? Despite the remarkable progress in robotic manipulation driven by generative models, deploying these policies in dynamic environments remains a formidable challenge. A predominant paradigm, Diffusion Policies, has demonstrated exceptional capability in modeling action space by leveraging Gaussian noise to represent robotic behaviors [ 9 ] . By learning the gradient of the data distribution, these methods can generate high-fidelity trajectories that capture human-like dexterity [ 37 ] . However, this expressiveness comes at a steep computational cost. The inherent mechanism of diffusion models relies on an iterative reverse denoising process. As highlighted in recent analyses [ 63 , 48 ] , this sequential computation introduces prohibitive inference latency, rendering standard diffusion approaches ill-suited for high-frequency, responsive manipulation tasks where real-time feedback loops are critical. To circumvent the latency bottleneck of iterative solvers, recent advances in Vision-Language-Action (VLA) models and Transformer-based architectures advocate for direct, end-to-end action generation [ 6 , 63 , 51 ] . These methods typically tokenize visual observations and directly predict actions in a feed-forward manner, ensuring rapid inference. However, achieving robust generalization with these architectures has historically demanded a brute-force strategy: scaling model parameters to billions and training on massive, diverse datasets. While effective at scale, this approach is data-inefficient and computationally heavy. Crucially, these methods often treat visual inputs as flattened sequences of semantic tokens, ignoring the intrinsic spatiotemporal mechanisms of manipulation. They lack explicit inductive biases to capture the fine-grained spatial relationships between the robotic end-effector and the target object, and often fail to distinguish task-relevant features from the dense, noisy visual context [ 45 , 29 ] . Consequently, without massive data covering every variation, these black-box mappings are prone to latching onto spurious background correlations, leading to poor performance under domain shifts or in cluttered scenes [ 23 ] . Thus, a critical research gap exists between the need for expressive, real-time control and the requirement for data-efficient generalization. The challenge lies in moving beyond simple parameter scaling or inefficient iterative denoising. Instead, we must explore how to construct an architecture that naturally embeds the spatiotemporal structure of manipulation tasks. This leads to the other core research question: II) How can robots effectively learn robust data-efficient policies from limited demonstrations? To surmount the deficiency in spatial-geometric reasoning outlined in I) , inspired by the efficiency of human cognitive

Read the complete text