High-fidelity 3D mesh generation from a single sketch using shape constraints Authors: Yingbin Wu, Fubo Wang, Peng Zhao, Mingquan Zhou, Shengling Geng, Dan Zhang Published: 2025 ABSTRACT The research on 3D model reconstruction from a single image using deep learning technology has achieved remarkable progress. However, compared with images, sketches lack sufficient visual information, which challenges the reconstruction algorithm’s ability to correctly interpret sketches. Herein, we introduce a streamlined network architecture for sketch-to-3D mesh generation, designed to address the challenge of reconstructing high-fidelity 3D models from single-hand sketches. Our approach deploys the expressive PowerMLP architecture within an encoder-decoder framework, surpassing traditional MLP implementations in representation capability. By integrating 3D shape constraints instead of relying on conventional discriminators, we achieve geometric fidelity in a collaborative generation process. Experimental results demonstrate state-of-the-art (SOTA) performance on both synthetic stylized sketches and real-world handwritten inputs, validating the method’s robustness and adaptability. Introduction Driven by the rapid evolution of computer graphics and VR/AR technologies, there has been a significant increase in the demand for 3D content creation across various domains. Sketch-based 3D reconstruction has emerged as an intuitive and efficient design paradigm, attracting considerable attention from both academic and industrial communities. This approach aims to rapidly generate 3D models from simple hand-drawn sketches, thereby reducing the technical complexity associated with traditional 3D modeling techniques and addressing the growing need for 3D content in fields such as film and television animation, game development, industrial design, and architectural design1–3. Current methods for sketch-based 3D modeling can be broadly categorized into three groups. The first group employs 3D model retrieval techniques, which match user-provided sketches against pre-existing 3D model libraries to generate the most similar 3D shapes3–6. The second group leverages multi-view 2D sketches and utilizes deep neural networks to reconstruct 3D shapes directly from these sketches7–11. More recently, cross-modal generation techniques have gained prominence by integrating sketch-based and text-to-3D generation methods, thereby expanding the scope and flexibility of 3D content creation12–14. However, these existing methods suffer from several limitations that hinder their widespread adoption. For instance, methods relying on 3D model retrieval often lack the ability to customize the retrieved models, thereby limiting the creative freedom of users. Other approaches that require multi-view sketches demand users to provide input sketches from multiple perspectives, which can be challenging for those without professional drawing skills. Similarly, methods integrating text prompts and sketch images also pose significant difficulties for non-expert users, as they necessitate both textual and visual inputs, further complicating the modeling process. Moreover, sketches are fundamentally different from real images. Composed primarily of simple lines, sketches lack effective visual information such as texture and lighting cues, which are crucial for algorithms to accurately interpret the intended 3D geometry15–17. The paucity of visual cues constitutes a formidable obstacle to reliable 3D-shape inference from sketches, necessitating algorithms that are both robust and contextually aware. To address the aforementioned challenges, we propose a novel end-to-end neural network architecture, termed Single View Sketch-to-Mesh (SS2M), designed to provide an intuitive and efficient 3D model reconstruction framework for users. The SS2M network enables rapid and effective generation of customized, high-fidelity 3D mesh models from a single input sketch that captures the user’s conceptual intent. In the SS2M network, the input sketch is first encoded into viewpoint and shape representations within a latent space via an encoder module. Subsequently, these latent encodings are decoded into a 3D mesh model through a decoder module, as illustrated in Fig. 1. This streamlined architecture allows users to bypass the complexities associated with multi-view sketching or additional textual inputs, thereby significantly lowering the technical barriers for non-expert users while maintaining high reconstruction quality. Fig. 1The architecture of SS2M network(The 3DViewer18 is used to render the 3D models displayed in the figure). The architecture of SS2M network(The 3DViewer18 is used to render the 3D models displayed in the figure). The SS2M network is designed as an encoder-decoder architecture with a streamlined structure. It incorporates PowerMLP19 neural network technology, which exhibits superior expressive capabilities compared to conventional MLP. Building upon the strengths of CNNs20, the SS2M encoder is capable of transforming sketch data from the input image space into a more compact and expressive low-dimensional feature representation. This enables the network to effectively capture complex shape patterns and spatial structures within sparse sketches. During the decoding process, the low-dimensional latent encoding is mapped into 3D shape data with depth information, facilitating the reconstruction of high-fidelity 3D models. Given that traditional encoder-decoder generative networks often struggle to produce high-quality 3D shapes, and considering that discriminator-based methods15,16 are prone to convergence difficulties during training21, we introduce 3D shape constraints to replace the conventional discriminator network. This innovation serves a dual purpose: it simplifies the overall network architecture while enforcing the generation of more realistic 3D shapes by directly constraining the output of the generation network. This approach ensures that the generated 3D meshes are geometrically accurate and faithful to the input sketch, thereby enhancing the overall performance and robustness of the SS2M. In summary, our approach has achieved superior 3D generation results through a more concise network architecture design. Specifically, our primary contributions are summarized as follows: We propose an end-to-end network model based on an encoder-decoder architecture with a streamlined structure. This model allows users to input a single-view sketch that captures their conceptual intent and rapidly generate customized, high-fidelity 3D mesh models with high efficiency.To facilitate the encoding-decoding process from image space to geometric space, we introduce PowerMLP neural network technology, which exhibits stronger expressive capabilities compared to traditional MLP. This enhancement enables the SS2M network to achieve superior generation performance within a comparable training timeframe.During network training, we incorporate 3D shapes from real datasets to compute similarity with the generated 3D models, serving as shape constraints to guide the network’s output. This approach enforces the generation of realistic 3D shapes, thereby improving the accuracy and plausibility of the reconstructed models.Extensive experiments and evaluations conducted on Shapenet-synthetic and Shapenet-sketch datasets demonstrate that the SS2M can accurately interpret user input sketches and generate corresponding high-fidelity 3D mesh models with enhanced precision and fidelity. We propose an end-to-end network model based on an encoder-decoder architecture with a streamlined structure. This model allows users to input a single-view sketch that captures their conceptual intent and rapidly generate customized, high-fidelity 3D mesh models with high efficiency. We propose an end-to-end network model based on an encoder-decoder architecture with a streamlined structure. This model allows users to input a single-view sketch that captures their conceptual intent and rapidly generate customized, high-fidelity 3D mesh models with high efficiency. To facilitate the encoding-decoding process from image space to geometric space, we introduce PowerMLP neural network technology, which exhibits stronger expressive capabilities compared to traditional MLP. This enhancement enables the SS2M network to achieve superior generation performance within a comparable training timeframe. To facilitate the encoding-decoding process from image space to geometric space, we introduce PowerMLP neural network technology, which exhibits stronger expressive capabilities compared to traditional MLP. This enhancement enables the SS2M network to achieve superior generation performance within a comparable training timeframe. During network training, we incorporate 3D shapes from real datasets to compute similarity with the generated 3D models, serving as shape constraints to guide the network’s output. This approach enforces the generation of realistic 3D shapes, thereby improving the accuracy and plausibility of the reconstructed models. During network training, we incorporate 3D shapes from real datasets to compute similarity with the generated 3D models, serving as shape constraints to guide the network’s output. This approach enforces the generation of realistic 3D shapes, thereby improving the accuracy and plausibility of the reconstructed models. Extensive experiments and evaluations conducted on Shapenet-synthetic and Shapenet-sketch datasets demonstrate that the SS2M can accurately interpret user input sketches and generate corresponding high-fidelity 3D mesh models with enhanced precision and fidelity. Extensive experiments and evaluations conducted on Shapenet-synthetic and Shapenet-sketch datasets demonstrate that the SS2M can accurately interpret user input sketches and generate corresponding high-fidelity 3D mesh models with enhanced precision and fidelity. Related works 3D reconstruction techniques 3D model reconstruction is a vital research direction within the field of computer vision. By leveraging computational methods to reconstruct the shape, structure, texture, and other attributes of objects in three dimensions, this technology enables multi-dimensional understanding and visualization of objects. It has garnered extensive attention and found wide application across various domains, including medical health, industrial production, cultural heritage preservation, game development, and virtual reality22,23. Existing 3D reconstruction technologies can be broadly categorized into three types: (1) Manual geometric modeling using software tools, which involves the creation of 3D models through manual design and manipulation in specialized software environments. (2) 3D scanning-based reconstruction, where devices such as laser radar or structured-light cameras are employed to capture the geometry of objects, and the resulting data is processed to generate 3D models. (3) Image-based 3D reconstruction, which utilizes single or multiple images and employs computer vision techniques to infer and reconstruct the 3D structure of objects from these images. Among these approaches, image-based 3D reconstruction has gained significant popularity in practical applications due to its low cost, ease of operation, and high fidelity in reconstruction outcomes22,23. Single-view image-based 3D model reconstruction Single-view image-based 3D object reconstruction aims to recover the 3D shape of an object from a single input image. Unlike multi-view 3D reconstruction, which leverages multiple perspectives to infer geometric information, single-view reconstruction is inherently challenging due to the limited geometric and semantic information available from a single viewpoint. The task of restoring a 3D shape from a single image is fundamentally ill-posed, as it involves significant ambiguity in depth and spatial relationships24,25. Early approaches to single-view 3D reconstruction primarily relied on visual cues such as lines, shadows, and textures within the image to make geometric assumptions and reconstruct the 3D surface of the object24,25. However, these methods are often domain-specific and lack generalizability, resulting in poor scalability and limited applicability to diverse object classes and scenarios. With the advent of large-scale 3D model datasets11,26,27 and the rapid advancement of deep learning techniques, the landscape of single-view 3D reconstruction has significantly evolved. Notably, the emergence of differentiable rendering technology in recent years28–31 has enabled the integration of rendering processes into end-to-end learning frameworks. This development, combined with the availability of extensive datasets, has facilitated the training of robust models capable of adapting to various complex variations in object shapes, appearances, and viewpoints15,22,23,28,31–37. Consequently, single-view 3D object reconstruction has become increasingly viable through large-scale data-driven approaches, paving the way for more accurate and generalizable solutions. Single-view sketch-based 3D model reconstruction Single-view sketch-based 3D modeling has long been a topic of active research in the field of computer graphics and vision. Existing methods can be broadly categorized into two types: end-to-end methods and interactive methods. Interactive methods rely on the decomposition of sketch drawing into multiple steps or the use of specific gestures38,39. These approaches require users to possess a certain level of strategic knowledge and expertise, which may pose challenges for non-professional users. On the other hand, end-to-end methods based on templates or retrieval can produce satisfactory results but often lack the flexibility for customization, thereby limiting their applicability in scenarios where personalized designs are desired. In recent years, there has been a growing interest in leveraging deep learning-based end-to-end methods to directly reconstruct 3D models from sketches5,8,12,15–17. This approach treats the task as a single-view 3D reconstruction problem, aiming to bridge the gap between sketch inputs and 3D outputs in a more seamless and automated manner. By harnessing the power of deep neural networks, these methods have shown promise in generating high-quality 3D models directly from sketches, thereby offering a more efficient and accessible solution for users. Sketch-based modeling is intrinsically challenging compared to traditional single-view 3D reconstruction, due to the abstraction and sparsity of sketch information. Sketches are characterized by their simplicity and lack of detailed visual cues such as texture, lighting, and shadows, which are essential for inferring 3D geometry. This scarcity of information makes the task of generating high-quality 3D shapes from sketches particularly demanding. In this paper, we address the problem of reconstructing single-view 3D objects from single-view sketches. Building on the work in15,28, we propose a streamlined end-to-end neural network architecture based on an encoder-decoder framework. Our method reconstructs a 3D mesh model from a single 2D sketch of the input object by leveraging PowerMLP technology and 3D shape constraints. This integrated approach effectively improves the accuracy and plausibility of the generated 3D models. Method We first establish a general baseline method for 3D model generation from a single-view sketch, as introduced in28. Building upon this baseline, we propose our SS2M network, which is specifically designed to produce 3D mesh model from a single-view sketch. Baseline Fig. 2The architecture of baseline method(The 3DViewer18 is used to render the 3D models displayed in the figure). The architecture of baseline method(The 3DViewer18 is used to render the 3D models displayed in the figure). As illustrated in Fig. 2, the input single sketch, denoted as I, is encoded into a shape code \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$Z=E(I)$$\end{document} in the latent space via an encoder comprising the CNN20 and MLP. The shape code Z is then passed through a decoder, D, to generate a 3D mesh model, \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$M=D{\mathrm{(}}Z{\mathrm{)}}$$\end{document}, that closely resembles the input sketch I. The generated mesh model M is composed of vertices and faces, and can be represented as \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$M={\mathrm{(}}{F_M},{V_M}{\mathrm{)}}$$\end{document}, where \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${V_M}$$\end{document} denotes the vertex vector and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${F_M}$$\end{document} represents the face vector in the mesh model. During the training process, the generated mesh model M is rasterized by the rasterizer R28 under a specified view angle V to produce a rendered grayscale silhouette image \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${S_r}$$\end{document}. This rendered image \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${S_r}$$\end{document} is then compared with the ground-truth rendered image \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${S_{gt}}$$\end{document} corresponding to the input sketch at the same view V to perform loss calculation. Finally, the parameters of the generator network are updated through error backpropagation to optimize the network performance. The aforementioned process can be mathematically formulated as follows:1\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$M=D(E(I))$$\end{document}2\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${S_r}=R(M,V)$$\end{document} It is important to note that the shape decoder D does not directly decode Z into the 3D mesh model M, but rather into the biases of the vertex vector \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${V_M}$$\end{document}. This is because the output mesh M is constructed by deforming an isotropic sphere with a predefined set of K-vertices. Assuming that the vertex vector of the original K-vertex sphere is denoted as \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$B{\mathrm{(}}{F_{b}}{V_b})$$\end{document}, the generated mesh M also comprises K vertices. The \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${F_M}$$\end{document} remains as the \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${F_{b}}$$\end{document}, but the \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${V_M}$$\end{document} is calculated as follows:3\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${V_M}={V_b}+D(Z)$$\end{document} The SS2M network architecture The SS2M builds upon the baseline model described in Sect. 3.1 by incorporating a viewpoint prediction branch as in Sketch2Model15. This branch predicts the corresponding perspective information while generating the 3D model, thereby guiding the sketch generation process to determine the shape under viewpoint constraints. However, unlike Sketch2Model, we have streamlined the network architecture, eschewing additional complex branch designs in favor of a more concise structure. Furthermore, we have integrated PowerMLP19 technology into the encoder-decoder framework to enhance the network’s expressive capabilities. Regarding the design of loss functions, conventional methods such as those in15,16,37 typically focus solely on the projection error loss between the 2D silhouette image of the generated 3D model and the input sketch. However, it is widely recognized that single-view sketches lack substantial 3D information, making it challenging to produce high-quality 3D shapes using only silhouette information for constraint and supervision. To address this limitation, we introduce a 3D shape constraint mechanism in SS2M, which directly calculates the 3D difference between the predicted shape and the ground-truth shape. This difference is incorporated as an additional term in the loss function, thereby ensuring that the generative network produces high-fidelity 3D results. In the following section, we will elaborate on the working principle of the SS2M network in conjunction with Fig. 1. Overall architecture Assuming the input single sketch is denoted as I, and its corresponding output mesh is represented as \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$M=({F_M},{V_M})$$\end{document}, where \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${F_M}$$\end{document} and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${V_M}$$\end{document} denote the face and vertex vectors of the mesh, respectively. Initially, we extract image features from the sketch I using the encoder E and map them into two separate latent spaces: the viewpoint latent space and the shape latent space. This process yields the viewpoint encoding \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${Z_v}$$\end{document} and the shape encoding \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${Z_s}$$\end{document}. Mathematically, this can be expressed as:4\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$({Z_v},{Z_s})=E(I)$$\end{document} The viewpoint decoder \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${D_v}$$\end{document}​ decodes \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${Z_v}$$\end{document} into a predicted viewpoint vector \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${V_p}={D_v}({Z_v})$$\end{document}. The loss between \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${V_p}$$\end{document} and the ground-truth viewpoint V in the sketch is then calculated. Meanwhile, the shape decoder \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${D_s}$$\end{document}​ decodes \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${Z_s}$$\end{document} into a predicted 3D mesh model M. The vertex vector \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${V_M}$$\end{document} and face vector \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${F_M}$$\end{document} in M are computed as the baseline mentioned in Sect. 3.1. The generated mesh M is subsequently compared with the ground-truth 3D model \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${M_{gt}}$$\end{document} corresponding to the sketch using a 3D shape similarity function, thereby achieving constraint control over the 3D shape fidelity. Meanwhile, under the constraint of the viewpoint V, the mesh M is rendered into a silhouette image \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${S_r}=R(M,V)$$\end{document} via a differentiable renderer R. The rendered silhouette image \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${S_r}$$\end{document} is then compared with the ground-truth rendered silhouette image \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${S_{gt}}$$\end{document} corresponding to the viewpoint V to compute the loss. A detailed introduction to the loss function will be provided in Sect. 3.4. Encoder and decoder The architecture of the encoder E is a hybrid of CNN20 and PowerMLP19, leveraging the strengths of both components. Specifically, the CNN is employed to extract salient feature information from the input sketch, focusing on essential stroke shape features while minimizing the influence of non-essential information such as painting style. Building on this foundation, two independent PowerMLP branches are utilized to map the extracted image spatial features into the viewpoint space and shape space, respectively, thereby facilitating the encoding of \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${Z_v}$$\end{document} and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${Z_s}$$\end{document}. For the decoding process, two separate PowerMLP branches are employed to decode \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${Z_v}$$\end{document} and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${Z_s}$$\end{document} independently. Specifically, the viewpoint latent space encoding \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${Z_v}$$\end{document} is decoded into the viewpoint vector \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${V_p}$$\end{document}, while the shape latent space encoding \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${Z_s}$$\end{document} is decoded into the vertex vector \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${V_M}$$\end{document} of the generated mesh M. PowerMLP Recently, a novel neural network architecture known as the Kolmogorov Arnold Network (KAN)40 has garnered significant attention from researchers. Grounded in the Kolmogorov-Arnold representation theorem, KAN aims to enhance the approximation capabilities and interpretability of neural networks through an innovative mathematical design. Unlike traditional MLP, KAN introduces a unique architectural feature by placing activation functions on the edges (i.e., weights) of the network rather than on the nodes. This distinctive design enables KAN to approximate complex functions with a reduced number of parameters, which are typically parameterized using B-spline functions, thereby offering greater flexibility. KAN has demonstrated remarkable performance in various tasks, including function fitting and solving partial differential equations. Recently, an increasing number of scholars have applied KAN or its variants to the field of computer vision, such as image and graphic classification and segmentation19,41–44. These applications have effectively validated the efficacy of KAN’s architecture, highlighting its potential for broader adoption in related domains. However, the training process of KAN is significantly more complex and considerably slower than that of the traditional MLP19,45,46. The primary bottleneck stems from the recursive iteration process involved in calculating spline functions. In response to this challenge, PowerMLP, as a novel neural network architecture, employs a simplified representation of spline functions. By introducing a non-iterative spline function representation based on ReLU power, PowerMLP circumvents the recursive calculations required in KAN and completes the process through direct matrix operations. This approach not only retains the expressive advantages of KAN but also substantially reduces computational complexity. The k-th power of ReLU is defined as follows:5\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\sigma _k}={({\mathrm{ReLU}}({\mathrm{0}},x))^k}$$\end{document} PowerMLP is an L-layer neural network defined as Eq. (6):6\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\mathrm{PowerMLP}}(x)=({\Psi _{L - 1}} \circ \cdots \circ {\Psi _1} \circ {\Psi _0})(x)where:{\Psi _l}({x_l})=\left\{ {\begin{array}{*{20}{c}} {{\alpha _l}b({x_l}){\sigma _k}({\omega _l}{x_l}+{\gamma _l}),{\text{ for }}l