Point2Primitive: CAD Reconstruction from Point Cloud by Direct Primitive Prediction

Xinzhu Ma, Cheng Wang, Chen Tang, Bin Wang, Shixiang Tang, Yuan Meng, Yunhong Wang, Di Huang · 2025

Read Point2Primitive: CAD Reconstruction from Point Cloud by Direct Primitive Prediction by Xinzhu Ma, Cheng Wang, Chen Tang, Bin Wang, Shixiang Tang, Yuan Meng, Yunhong Wang, Di Huang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-02043.txt.

Introduction

Xinzhu Ma and seven colleagues wrote a 2025 computer vision paper on rebuilding CAD models from point clouds. A point cloud is a spray of dots that traces an object's surface. Turning that spray back into an editable design file is a core task in engineering.

Common methods hide the design inside a smooth field, which blurs sharp edges and makes edits hard. Their system, Point2Primitive, predicts the design's basic shapes directly. A transformer decoder treats the shape set as one prediction task. Each primitive stays exact and parametric, ready for a designer to tweak. Tests judge how close the rebuilt models come to the originals.

Catalog record

CREATORXinzhu Ma, Cheng Wang, Chen Tang, Bin Wang, Shixiang Tang, Yuan Meng, Yunhong Wang, Di Huang
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE45 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-02043

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Point2Primitive: CAD Reconstruction from Point Cloud by Direct Primitive Prediction Xinzhu Ma, Cheng Wang, Chen Tang, Bin Wang, Shixiang Tang, Yuan Meng, Yunhong Wang, Di Huang Abstract Recovering CAD models from point clouds requires reconstructing their topology and sketch-based extrusion primitives. A dominant paradigm for representing sketches involves implicit neural representations such as Signed Distance Fields (SDFs). However, this indirect approach inherently struggles with precision, leading to unintended curved edges and models that are difficult to edit. In this paper, we propose Point2Primitive, a framework that learns to directly predict the explicit, parametric primitives of CAD models. Our method treats sketch reconstruction as a set prediction problem, employing a improved transformer-based decoder with explicit position queries to directly detect and predict the fundamental sketch curves (i.e., type and parameter) from the point cloud. Instead of approximating a continuous field, we formulate curve parameters as explicit position queries, which are optimized autoregressively to achieve high accuracy. The overall topology is rebuilt via extrusion segmentation. Extensive experiments demonstrate that this direct prediction paradigm significantly outperforms implicit methods in both primitive accuracy and overall geometric fidelity. * * footnotetext: Corresponding Author 1 Introduction Computer-Aided Design (CAD) reconstruction, which aims to convert unstructured data like point clouds into structured, parametric models, is a long-standing goal in computer graphics and engineering ( Kruth and Kerstens, 1998 ) . Among various modeling techniques, the sketch-and-extrude process is one of the most fundamental operations, capable of generating a vast array of mechanical parts and industrial designs ( Willis et al., 2021 ) . Therefore, automatically reversing this process, transforming a raw 3D scan into a structured sketch-based CAD representation, holds immense value for digital manufacturing, legacy part recreation, and design modification. Despite recent progress driven by large-scale datasets ( Willis et al., 2021 ; Koch et al., 2019 ) , a critical challenge lies in accurately capturing the model’s boundaries. The unstructured, noisy nature of point clouds contrasts sharply with the precise, topologically-defined structure of CAD models. A dominant paradigm in recent literature has been to represent 2D sketches using implicit neural representations such as Signed Distance Fields (SDFs) ( Uy et al., 2022 ; Ren et al., 2022 ; Li et al., 2023a ) . While these methods are powerful for capturing complex, continuous surfaces, they remain limited in several critical aspects. First, SDFs are purely geometric fields that lack semantic structure, offering no direct correspondence to the primitive entities used in CAD. Second, as approximation-based representations, they inevitably introduce noise and blur fine geometric details, which makes it difficult to recover sharp corners or well-defined curves. Consequently, reconstructed sketches often exhibit smoothed or rounded edges rather than precise boundaries. Finally, because SDF-based sketches cannot be directly expressed in terms of editable primitives— lines, circles, and arcs , the lingua franca of CAD software—an additional conversion step is required. This transformation not only complicates the workflow but also accumulates parameter errors, further hindering faithful reconstruction and downstream editability. The above limitations lead us to a fundamental question: Can we bypass the imprecise, intermediate representations and directly predict the explicit, parametric sketch primitives from the point cloud? In this work, we affirmatively answer this question by proposing Point2Primitive , a framework grounded in a direct prediction methodology for CAD reconstruction from point clouds. Unlike indirect approaches that rely on intermediate surface fitting or voxelization, our method directly targets the underlying parametric sketch primitives . Specifically, we observe that many CAD models can be systematically deconstructed into a sequence of extrusion operations, each defined by a two-dimensional sketch—a closed profile formed by geometric primitives such as lines, arcs, and circles—together with an associated three-dimensional operation that specifies the extrusion direction and length. Building on this insight, we reformulate the complex reverse engineering problem into two interdependent sub-tasks. First, in the stage of topology reconstruction , the input point cloud is partitioned into clusters that correspond to individual extrusions, a process we term extrusion segmentation . This step disentangles the overall geometry into meaningful sub-structures, enabling subsequent sketch prediction to operate on localized and semantically coherent point sets. Second, in the stage of primitive prediction , we recover the generating two-dimensional sketch for each segmented extrusion body. To achieve this, we design an improved transformer-based decoder that refines point features and directly decodes them into parametric primitives. The decoder integrates explicit geometric priors by formulating curve parameters as positional queries, which guide the attention mechanism and progressively update predictions across layers. This design enhances the accuracy and stability of primitive recovery. Point2Primitive could bridge topology recovery and parametric sketch inference within a unified direct-prediction framework. By directly predicting the language of CAD, our method produces CAD models that are not only geometrically accurate but also truly editable. In summary, our main contributions are summarized as follows: • We introduce Point2Primitive, a new pipeline that directly predicts parametric sketch primitives (i.e., lines, arcs, and circles) from raw point clouds, bypassing implicit fields and preserving sharp, semantically meaningful boundaries for downstream editability. • We decompose reconstruction into (i) extrusion segmentation to recover topology by clustering the point cloud into extrusion bodies, and (ii) primitive prediction with an improved transformer decoder that uses explicit position queries and autoregressive parameter refinement to yield high-precision primitive types and parameters. • Extensive experiments and ablations show consistent gains in boundary fidelity, primitive recovery, and editability over SDF-based and recent learning baselines, demonstrating both accuracy and robustness across diverse CAD geometries. 2 Related works 2.1 CAD Reconstruction via Primitive Detection Several learning-based approaches have also been proposed to fit geometric primitives to point clouds. These methods first segment the input points into clusters corresponding to the same surface primitives and fit the clustered points to the parametric surface primitives ( Li et al., 2019 ; Sharma et al., 2020 ; Le et al., 2021 ; Li et al., 2023b ) or free surfaces ( Liu et al., 2023 ) . Except for surface primitives, UCSG-Net ( Kania et al., 2020 ) and CSG-stump ( Ren et al., 2021 ) learn the CSG parse tree that interrelates the predicted geometry primitives to reconstruct shapes. The reconstructed CAD models are in B-rep format, which is not as easy to edit as the sketch-and-extrude modeling procedure. In this paper, we explore the inverse of the sketch-and-extrude process and strive to recover the CAD modeling procedure from the point cloud. Therefore, we segment the point clouds into clusters belonging to the same extrusion primitive and explicitly reconstruct every element of the extrusion primitives. Instead of fitting points to the parametric or free surface, we employ an improved transformer to learn and predict the curves directly. 2.2 Sketch-and-extrude CAD Reconstruction Some new solutions use SDF to represent sketches following the recent implicit 3D shape representations ( Park et al., 2019 ; Wang et al., 2022 ) . ExtrudeNet ( Ren et al., Conference Proceedings ) and SECAD-Net ( Li et al., 2023a ) can learn implicit sketches and differentiable extrusions. Point2Cyl reconstructs 3D shapes through sketch regression supervised by the latent embeddings of its SDF. However, these methods utilize the implicit fields for sketch representation, leading to curved edges of the reconstructed shapes. Meanwhile, the reconstructed sketch is hard to be

Read the complete text