Multimodal Spiking Neural Network for Space Robotic Manipulation
2025
Read Multimodal Spiking Neural Network for Space Robotic Manipulation (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-07287.txt.
Introduction
Liwen Zhang and co-authors present a multimodal spiking neural network for space robotic manipulation in a 2025 paper. The hosted head lists no byline, and the catalog names these five authors. Space station arms must work on tight onboard resources.
The control frame blends shape data with touch and semantic inputs for stronger scene awareness. Spiking networks, which fire in pulses, suit low-power hardware. A dual-channel, three-stage curriculum teaches skills step by step. Tests covered target approach, grasping, and stable lifting with wall-mounted arms. The method beat baselines on success rate and energy use. The authors pitch it as fit for real aerospace operations where power and compute are scarce.
Catalog record
| YEAR | 2025 |
|---|---|
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 16 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2508-07287
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Multimodal Spiking Neural Network for Space Robotic Manipulation Abstract This paper presents a multimodal control framework based on spiking neural networks (SNNs) for robotic arms aboard space stations. It is designed to cope with the constraints of limited onboard resources while enabling autonomous manipulation and material transfer in space operations. By combining geometric states with tactile and semantic information, the framework strengthens environmental awareness and contributes to more robust control strategies. To guide the learning process progressively, a dual-channel, three-stage curriculum reinforcement learning (CRL) scheme is further integrated into the system. The framework was tested across a range of tasks including target approach, object grasping, and stable lifting with wall-mounted robotic arms, demonstrating reliable performance throughout. Experimental evaluations demonstrate that the proposed method consistently outperforms baseline approaches in both task success rate and energy efficiency. These findings highlight its suitability for real-world aerospace applications. Liwen Zhang 1 , Dong Zhou 1 , Shibo Shao 1 , Zihao Su 1 , Guanghui Sun* 1 1 School of Astronautics, Harbin Institute of Technology, Harbin, China Keywords: Space Robotics, Spiking Neural Networks, Curriculum Reinforcement Learning, Multimodal Perception Nomenclature d = the Euclidean distances from the gripper center to the target object d_{\text{lf}},d_{\text{rf}} = the Euclidean distances from the left fingertip and right fingertip to the target object \mathbf{p}_{\text{mid}}^{xz},\mathbf{p}_{\text{obj}}^{xz} = the projected position of the gripper center and the target object onto the X–Z plane y_{\text{lf}},y_{\text{rf}} = the Y-coordinates of the left and right fingertips y_{\text{mid}},y_{\text{obj}} = the Y axis positions of the gripper center and the target object \delta_{i},\lambda_{i},\alpha_{i},\gamma_{i} = the tunable parameters \epsilon = a safety margin accounting for tolerable deviation in finger spacings \text{gap}_{\text{eef}} = the Euclidean distance between the two fingertips s_{\text{obj}} = the side length of the target object \mathbf{f}_{\text{lf}},\mathbf{f}_{\text{rf}} = the sensed contact forces at the left and right fingertips \mathbf{d}_{\text{lf}\rightarrow\text{obj}},\mathbf{d}_{\text{rf}\rightarrow\text{obj}} = the normalized pointing direction from each fingertip to the target object \mathbf{1}_{\text{lf\_t}},\mathbf{1}_{\text{rf\_t}} = binary signals indicating fingertip contact (1 if in contact, 0 otherwise) \mathbf{y}_{\text{eef}} = the end-effector’s Y axis \mathbf{y} = the world Y axis h_{0},h_{1},h_{2} = h_{0} is object height, h_{1} , h_{2} are lift thresholds s_{i}^{(b)} = whether neuron i fired in sample b V_{i}(t) = the membrane potential of neuron i at time t \mathbb{I}(\cdot) = the indicator function B = the batch size T = the simulation steps N_{0},N_{1},N_{2} = the layer sizes x_{i}^{(b)},h_{j}^{(b)} = the activations of input and output neurons \mathbf{f}_{i} = the 3D force vector measured at fingertip i f_{ix},f_{iy},f_{iz} = the components of the contact force along the X, Y, and Z coordinate axes \mathbf{p}_{i} = the fingertip position \mathbf{c} = the object’s center of mass 1 Introduction As crewed spaceflight, Mars exploration, and on-orbit servicing advance, space manipulators have become essential to spacecraft operations. They are generally categorized as extravehicular or intravehicular [ 1 ] . Extravehicular manipulators, with extended reach and high robustness, serve in tasks such as assembly, maintenance, and target capture [ 2 , 3 , 4 ] . Intravehicular manipulators are adapted to confined environments and enable fine manipulation within spacecraft cabins [ 5 , 6 ] . Conventional control approaches rely on accurate modeling and vision-based guidance [ 7 , 8 , 9 ] . Recently, artificial neural networks (ANNs) with reinforcement learning (RL) has attracted increasing attention for space manipulator control [ 10 , 11 , 12 , 13 , 14 ] . However, ANN-based methods often involve high computational cost and latency, which limit their suitability for space missions requiring real-time performance and energy efficiency. Spiking neural networks (SNNs) have demonstrated promising energy efficiency in terrestrial robotics. However, their application in space manipulator control remains an open research question. Motivated by this gap, we develop a control framework that integrates SNNs with RL. The framework examines the feasibility of using SNNs for energy-efficient control, multimodal perception, and task execution under typical space-related constraints. The structure of this paper is as follows: Section 2 introduces the control approach. Section 3 presents the experiments and results. Section 4 concludes the work and discusses future directions. 2 Methodology 2.1 Model Architecture A control architecture for robotic manipulators in space station environments is designed in this study, inspired by the SpikeGym framework [ 15 ] . A schematic of the overall system is shown in the left part of Fig. 1 . The proposed control framework incorporates three categories of sensory input: physical states, tactile signals, and semantic cues. Environmental observations are encoded into spike trains by leaky integrate-and-fire (LIF) neurons at the input layer. These spike patterns are then processed by a non-spiking LIF (N-LIF) output layer, which generates signals for action selection and value estimation. The control process is structured in three stages. In the first stage, spatial features are integrated to guide accurate target approach. The second stage incorporates tactile force information and high-level semantic information, enabling the system to model and infer grasp affordance more effectively. In the third stage, the framework shifts from purely geometric representations to a multimodal learning mechanism, allowing the robot to adapt more robustly to dynamic and uncertain interactions. Finally, the system is trained in an end-to-end manner using proximal policy optimization (PPO) with generalized advantage estimation (GAE). Figure 1: The system architecture. 2.2 Tactile Feature Representation This work proposes a set of tactile features to characterize contact interactions during robotic grasping, motivated by the insights discussed in the review by [ 16 ] . Specifically, we define: (1) the norm of the contact force at each fingertip is used to quantify local contact intensity; (2) the symmetry of force magnitudes between the two fingertips indicates force balance; and (3) the geometric consistency is evaluated via the directional alignment between the applied contact force \mathbf{f}_{i} and the positional vector from the fingertip location \mathbf{p}_{i} to the object’s center of mass \mathbf{c} . The formal definitions of these features are provided below. \|\mathbf{f}_{i}\|=\sqrt{f_{ix}^{2}+f_{iy}^{2}+f_{iz}^{2}},~s=\left|\|\mathbf{f}_{\text{lf}}\|-\|\mathbf{f}_{\text{rf}}\|\right|,~\cos(\theta_{i})=\frac{\mathbf{f}_{i}}{\|\mathbf{f}_{i}\|}\cdot\frac{\mathbf{c}-\mathbf{p}_{i}}{\|\mathbf{c}-\mathbf{p}_{i}\|}. (1) 2.3 Training Method To address the instability induced by unexpected drift and rotation of target objects during contact in microgravity, we introduce a dual-channel, three-phase curriculum reinforcement learning (CRL) framework (see the right part of Fig. 1 ). The framework guides policy learning through staged modulation of reward structure and success criteria during training. The early stage emphasizes geometric alignment. As training progresses, the focus shifts toward tactile-guided control in the intermediate phase. In the final stage, the reward design centers on task completion as the primary objective. This structured curriculum enhances learning efficiency and improves policy robustness under space-relevant dynamic uncertainties. The reward structure consists of five components: geometric alignment, tactile feedback and grasp execution, lift execution under posture constraints, and penalty terms. (I) Exploration-Oriented The main reward terms are defined as follows. The geometric proximity reward ( r_{\text{geo}} ) encourages the end-effector to maintain spatial closeness to the target. The pose and symmetry regularization term ( r_{\text{sym}} ) promotes alignment accuracy. Additionally, the failure aversion penalty ( r_{\text{hover}} ) penalizes actions that result in ineffective or unstable behavior. r_{\text{geo}}=\sum_{i=1}^{3}\alpha_{i}\left(1-\tanh(\lambda_{i}\cdot d_{i})\right),d_{1}=\frac{d+d_{\text{lf}}+d_{\text{rf}}}{3},d_{2}=\left\|\mathbf{p}_{\text{mid}}-\mathbf{p}_{\text{obj}}\right\|,d_{3}=\left\|\mathbf{p}_{\text{mid}}^{xz}-\mathbf{p}_{\text{obj}}^{xz}\right\|. (2) r_{\text{sym}}=\alpha_{4}\left(1-\tanh\left(\lambda_{4}\cdot|y_{\text{lf}}-y_{\text{rf}}|\right)\right)+\gamma_{1}\cdot\max\left(0,\delta_{1}-|y_{\text{mid}}-y_{\text{obj}}|\right). (3) r_{\text{hover}}=-\exp\left(\lambda_{5}\cdot\left(|y_{\text{mid}}-y_{\text{obj}}|-\delta_{2}\right)\right). (4) (II) Skill Learning The tactile-grasp consistency reward ( r_{\text{t-g}} ) encourages balanced contact and stable grasping of the object. The condition for grasp success, transitioning progressively from a geometry-based assessment to one informed by