CLASP: Closed-loop Asynchronous Spatial Perception for Open-vocabulary Desktop Object Grasping

Yiran Ling, Wenxuan Li, Siying Dong, Yize Zhang, Xiaoyao Huang, Jing Jiang, Ruonan Li, Jie Liu · 2026

Read CLASP: Closed-loop Asynchronous Spatial Perception for Open-vocabulary Desktop Object Grasping by Yiran Ling, Wenxuan Li, Siying Dong, Yize Zhang, Xiaoyao Huang, Jing Jiang, Ruonan Li, Jie Liu (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-11320.txt.

Introduction

Ling and seven colleagues build CLASP, a closed-loop framework for open-vocabulary desktop object grasping by robots. A dual-pathway perception module separates high-level semantic intent from geometric grounding to curb spatial hallucination. An asynchronous evaluator compares states before and after each action and feeds back text diagnostics for error correction. The goal is grasping that survives clutter and odd shapes, not just clean lab scenes. A data engine synthesizes spatial labels and reasoning templates without a human driving the robot. Tests reach an 87.0 percent overall success rate.

Catalog record

CREATORYiran Ling, Wenxuan Li, Siying Dong, Yize Zhang, Xiaoyao Huang, Jing Jiang, Ruonan Li, Jie Liu
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE32397
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.11320

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

CLASP: Closed-loop Asynchronous Spatial Perception for Open-vocabulary Desktop Object Grasping Yiran Ling, Wenxuan Li, Siying Dong, Yize Zhang, Xiaoyao Huang, Jing Jiang, Ruonan Li, Jie Liu Abstract Robot grasping of desktop object is widely used in intelligent manufacturing, logistics, and agriculture.Although vision-language models (VLMs) show strong potential for robotic manipulation, their deployment in low-level grasping faces key challenges: scarce high-quality multimodal demonstrations, spatial hallucination caused by weak geometric grounding, and the fragility of open-loop execution in dynamic environments. To address these challenges, we propose Closed-Loop Asynchronous Spatial Perception(CLASP), a novel asynchronous closed-loop framework that integrates multimodal perception, logical reasoning, and state-reflective feedback. First, we design a Dual-Pathway Hierarchical Perception module that decouples high-level semantic intent from geometric grounding. The design guides the output of the inference model and the definite action tuples, reducing spatial illusions. Second, an Asynchronous Closed-Loop Evaluator is implemented to compare pre- and post-execution states, providing text-based diagnostic feedback to establish a robust error-correction loop and improving the vulnerability of traditional open-loop execution in dynamic environments. Finally, we design a scalable multi-modal data engine that automatically synthesizes high-quality spatial annotations and reasoning templates from real and synthetic scenes without human teleoperation. Extensive experiments demonstrate that our approach significantly outperforms existing baselines, achieving an 87.0% overall success rate. Notably, the proposed framework exhibits remarkable generalization across diverse objects, bridging the sim-to-real gap and providing exceptional robustness in geometrically challenging categories and cluttered scenarios. I INTRODUCTION With the continuous integration of artificial intelligence and robotics, enabling robots to perform intelligent grasping according to natural language instructions has become a core requirement for autonomous manipulation. In open and unstructured environments, grasping under desktop-like planar constraints is particularly important because objects are diverse, layouts change frequently, and interaction often occurs in close proximity to people. Such capabilities are crucial across many practical scenarios, including retail settings like supermarkets, office desktops and shared workspaces, and other everyday service contexts where robots must retrieve, organize, or deliver items safely and efficiently. Grasping in real-world scenarios requires robots to not only perceive the geometric structure of objects and ensure grasping stability, but also understand complex semantic constraints. Existing robotic grasping approaches can be mainly categorized into two paradigms: rule-based grasping methods and recent Learning-based grasping methods. Rule-based robotic grasping methods [ 5 , 4 ] typically rely on geometric information of objects and make insufficient use of the semantic information of the objects themselves. Based on learning methods such as vision-language models(VLM) and Visual-Language-Action(VLA) models [ 32 , 2 , 14 ] , it exhibits strong generalization ability for object manipulation. Although the Visual-Language-Action(VLA) models also perform well in grasping desktop objects, they are highly dependent on post-training. In recent years, the development of vision-language models (VLMs) [ 17 , 28 , 13 ] has introduced semantic understanding into grasping tasks. By incorporating affordance information into multimodal pre-training data, VLM can assist robots in grasping open vocabulary objects under natural language commands. Despite promising strides, deploying VLMs for low-level robotic manipulation faces critical bottlenecks: 1) a severe scarcity of high-quality multi-modal demonstration data; 2) VLMs’ lack of fine-grained geometric grounding, leading to spatial hallucinations. To address these challenges, we propose a novel closed-loop framework that bridges semantic reasoning with physical execution. By combining hierarchical perception with asynchronous closed-loop feedback, spatial illusions in robot grasping based on visual language models can be alleviated, bridging the semantic and geometric gap. The main contributions of this paper are summarized as follows: • We introduce a dual-pathway hierarchical perception approach that decouples semantic intent from geometric grounding. By outputting structured action tuples ( e.g. , 2D contact points and angles), this method mitigates spatial hallucinations and enables deterministic control. • We design an asynchronous closed-loop evaluator. In this evaluator, a Judger module compares pre- and post-execution states and generates diagnostic feedback. This establishes a robust error-correction mechanism for improved task success in dynamic environments. • We develop a scalable multi-modal data engine. Within this engine, an automated pipeline synthesizes high-quality demonstration data (visual prompts, masks, grasp poses) without human teleoperation. II RELATED WORKS II-A Basic Grasp Detection Models Existing grasping research mainly optimizes performance along a single dimension, lacking a unified framework that integrates geometric, semantic, and adaptive capabilities. For example, GraspNet [ 5 ] and AnyGrasp [ 4 ] directly regress 6-DoF grasps from point clouds or RGB-D images, achieving accurate geometric evaluation but ignoring semantic cues. [ 18 , 11 , 29 ] enhance stability via geometric modeling or tactile feedback, yet rely on complex designs or specialized hardware. Task-aware approaches [ 27 , 19 , 10 ] model gestures or manipulation relationships to capture intent, but require extra modalities and show limited cross-scenario generalization. Moreover, environment- or hardware-specific optimizations [ 16 , 24 , 7 , 6 ] improve robustness in constrained settings, while remaining scenario- or task-dependent. II-B Learning-based Grasping and Manipulation Learning-based grasping methods leverage large models for semantic understanding and reasoning in open-world scenarios. Early works [ 23 , 15 ] adopt contrastive learning for multi-modal alignment but lack complex instruction reasoning. [ 17 , 28 , 13 ] introduce large-scale pre-training frameworks linking visual encoders and language models via projection layers, enabling instruction following and visual reasoning, yet remain unoptimized for robotic interaction. Building on this, several studies directly apply vision-language models (VLMs) to grasping. Methods in [ 26 , 25 ] reformulate affordance recognition as VLM-based localization, enabling semantic-guided ROI generation but suffering from error accumulation in two-stage designs. [ 22 ] incorporates task-level reasoning for cluttered environments, improving strategic planning at the cost of real-time performance. End-to-end VLA models [ 32 , 2 ] like OpenVLA [ 14 ] , directly input vision-language inputs into robot actions and achieve generalizable task execution capabilities, but require extensive fine-tuning data and large model scales, limiting their effectiveness. deployment in long-tail and resource-constrained settings.Recent work increasingly explores dexterous manipulation for complex tasks, such as DexVLG [ 9 ] and DexGrasp [ 30 ] . However, high hardware costs, limited compatibility with common configurations, and the lack of standardized datasets hinder practical deployment. III Method III-A Problem Formulation Fig. 1: Overview of the functional components of the CLASP framework: The figure shows the core modules, from natural language instruction input and dual-path awareness to VLM-based heuristic reasoning and closed-loop feedback mechanism. We define a closed-loop system \mathcal{S}=(\mathcal{P},\mathcal{R},\mathcal{E}) that maps an instruction \ell and observation I_{t} to an action a_{t} and a discrete state S_{t} , i.e., (a_{t},S_{t})=\mathcal{S}(\ell,I_{t}) . The pipeline contains a Dual-Pathway Perception module \mathcal{P} , a Vision-Language Reasoner \mathcal{R} , and a Visual-Semantic Evaluator \mathcal{E} . The iteration continues until a terminal condition S_{t}\in\mathcal{S}_{\text{term}} holds. Definition 1 Task-Oriented Grasp Objective : Given a visual observation I , we define a task-conditioned energy functional E(g;I) to evaluate any grasp g\in\mathcal{G} , where \mathcal{G} represents the feasible grasp space. Task-oriented grasping refers to finding the optimal grasp g^{\star} from this feasible set that minimizes the combined energy functional, namely, g^{\star}=\arg\min_{g\in\mathcal{G}}E(g;I), (1) subject to E(g;I)=\lambda_{g}E_{geo}(g;I)+\lambda_{s}E_{sem}(g;I), (2) where E_{geo}(g;I) measures geometric grasp stability (e.g., antipodal alignment, local surface compatibility), and E_{sem}(g;I) encodes functional affordance preferences (e.g., grasping the handle of a pan). Definition 2 Parallel-Jaw Grasp Parameterization : To operationalize the grasp space \mathcal{G} ,

Read the complete text