Easy3D: A Simple Yet Effective Method for 3D Interactive Segmentation
Andrea Simonelli, Norman Müller, Peter Kontschieder · 2025
Read Easy3D: A Simple Yet Effective Method for 3D Interactive Segmentation by Andrea Simonelli, Norman Müller, Peter Kontschieder (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-11024.txt.
Introduction
Andrea Simonelli, Norman Müller, and Peter Kontschieder simplify the job of picking objects out of 3D scenes with clicks, in work from 2025. Interactive 3D segmentation serves editing, robotics, and reconstruction, but tools must stay fast and work in unfamiliar settings.
Easy3D pairs a sparse voxel encoder with a small transformer decoder that folds user clicks into its features implicitly. The plain design beat earlier state-of-the-art systems on four standard benchmarks, indoors and on street scenes. It also held up on geometry from Gaussian Splatting, a distribution it was not trained on. The authors stress that simplicity, handled well, can outperform heavier machinery in both accuracy and efficiency.
Catalog record
| CREATOR | Andrea Simonelli, Norman Müller, Peter Kontschieder |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 42 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2504-11024
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Easy3D: A Simple Yet Effective Method for 3D Interactive Segmentation Andrea Simonelli, Norman Müller, Peter Kontschieder Abstract The increasing availability of digital 3D environments, whether through image-based 3D reconstruction, generation, or scans obtained by robots, is driving innovation across various applications. These come with a significant demand for 3D interaction, such as 3D Interactive Segmentation, which is useful for tasks like object selection and manipulation. Additionally, there is a persistent need for solutions that are efficient, precise, and performing well across diverse settings, particularly in unseen environments and with unfamiliar objects. In this work, we introduce a 3D interactive segmentation method that consistently surpasses previous state-of-the-art techniques on both in-domain and out-of-domain datasets. Our simple approach integrates a voxel-based sparse encoder with a lightweight transformer-based decoder that implements implicit click fusion, achieving superior performance and maximizing efficiency. Our method demonstrates substantial improvements on benchmark datasets, including ScanNet [ 3 ] , ScanNet++ [ 35 ] , S3DIS [ 1 ] , and KITTI-360 [ 17 ] , and also on unseen geometric distributions such as the ones obtained by Gaussian Splatting [ 12 ] . The project web-page is available here: https://simonelli-andrea.github.io/easy3d 1 Introduction Technological progress has increased the demand for applications that enhance understanding and interaction in digital 3D environments. These environments are in fact now easily obtainable through scene reconstruction ( e.g . , NeRFs [ 20 ] , Gaussian Splatting [ 12 ] ), scene generation, acquisition from laser scans ( e.g . , LiDAR) or sensors mounted on robots. Among the many tasks that enhance understanding and interaction with 3D environments, 3D Instance Segmentation is essential for enabling applications that involve object selection and manipulation. Figure 1 : Example of a typical use-case of Interactive 3D Instance Segmentation (top) and overview of our method’s components (bottom). A user interacts with a 3D scene (top left) and defines a 3D click (blue circle) to select an object (table). The scene and clicks are fed to Easy3D , which generates the corresponding instance segmentation mask (top right). After visually inspecting the result (red mask), the user can provide feedback and refine the output with additional 3D clicks. To obtain the mask, Easy3D initially maps the scene and user click(s) to a voxelized 3D representation, to then extract scene features using a sparse U-Net. The processed scene and clicks are then fed to a two-directional transformer decoder, which exchanges information through attention to update them. Finally, an implicit click fusion operation is used to predict the output instance segmentation mask. While many approaches have been explored in literature [ 8 , 36 , 6 , 10 , 31 , 24 , 14 , 32 , 26 , 9 ] , most are passive , offering a fixed set of outputs that may not fully meet user needs. Interactive methods, however, enable users to interact with a single 3D click to generate an initial segmentation mask. The result can be immediately observed and further refined by adding positive or negative clicks on the obtained masks, hence allowing for direct user interaction and feedback (as in Fig. 1 ). Current state-of-the-art architectures typically consist of an encoder and a transformer-based decoder, and can be broadly categorized based on their choice of encoder and click fusion strategy. The click fusion strategy is the method by which information from multiple user clicks, which can be positive or negative, is combined to predict a single object mask. Some methods rely on explicit click fusion, which struggles with out-of-domain and unseen objects (e.g., [ 37 ] ). Conversely, existing methods that use implicit click fusion (e.g., [ 39 ] ) often have inconsistent performance across datasets and rely on architectures (like ViT [ 5 ] ) that require a much higher number of parameters. As applications involving 3D understanding and interaction become more prevalent, it is desirable to have a solution that is simple, efficient, precise, but also capable of performing consistently well across diverse settings, including unfamiliar environments and objects. For these reasons we introduce Easy3D , which combines the strengths of all state-of-the-art approaches by utilizing a voxel-based encoder alongside implicit click fusion. This combination allows our method to achieve superior efficiency and generalization across datasets. Additionally, we incorporate a negative mask component, traditionally used in explicit click fusion, to help the model better understand background elements of the scene. We are the first to demonstrate the positive impact of integrating this component with implicit click fusion, further enhancing the method’s performance. By addressing the limitations of existing methods, our method offers an easy but yet robust solution for 3D instance segmentation, paving the way for more effective and adaptable applications in diverse environments. In summary, the contributions of our paper include: • A simple and efficient voxel-based, interactive 3D instance segmentation method, outperforming existing state-of-the-art methods on both seen and unseen domains, and across challenging benchmark datasets like ScanNet [ 3 ] , ScanNet++ [ 35 ] , S3DIS [ 1 ] , and KITTI-360 [ 17 ] . • The integration of a learned negative embedding in implicit click fusion, improving generalization and segmentation performance. • A targeted analysis on the impact of implicit and explicit click fusion in the context of 3D instance segmentation, providing insights w.r.t. performance and generalization. 2 Related Works Interactive methods are developed to correct outputs generated by a given system or model, typically through actively provided user feedback. The concept of interactivity has been extensively studied in the 2D setting, starting from pioneering works such as [ 11 , 21 ] to more recent approaches [ 34 , 16 , 19 , 13 , 23 ] , which propose image segmentation methods allowing user feedback via 2D clicks. Among these, Segment Anything [ 13 ] is the most widely known and adopted method, thanks to its versatile architecture that enables segmentation of any object in an image. Figure 2 : Architecture of our method for which we provide details in Sec. 3.2 . The set of user clicks C (top left) is encoded into the clicks embedding C_{E} . The input scene points S_{P} (bottom left) are mapped into a voxelized scene S_{V} and encoded into a scene embedding S_{E} . The clicks and scene embedding, with the additional learned embeddings L_{E} (e.g. output embeddings), are then fed to the decoder which uses attention operations to update them. The updated embeddings are then fused using a click fusion strategy to obtain the segmentation mask on voxels M_{V} , which is finally mapped back to the original scene points to obtain the output points mask M_{P} . In the 3D setting, the majority of the literature focuses on non-interactive methods. Early works such as [ 8 , 36 , 6 , 10 , 31 ] have recently evolved towards transformer-based architectures that leverage attention mechanisms [ 30 ] , including Mask3D [ 24 ] , OneFormer3D [ 14 ] , and PTv3 [ 32 ] . Additionally, works like OpenMask3D [ 26 ] and Segment3D [ 9 ] focus on open-set and label-free 3D segmentation, respectively. While many approaches exist for Interactive Semantic 3D Segmentation [ 25 , 38 , 15 , 29 ] , the