Shap-MeD

Nicolas Laverde 1, Melissa Robles 2, Johan Rodríguez 3 · 2025

Read Shap-MeD by Nicolas Laverde 1, Melissa Robles 2, Johan Rodríguez 3 (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-15562.txt.

Introduction

Nicolás Laverde, Melissa Robles, and Johan Rodríguez adapt in 2025 a text-to-3D generator for medicine. Typing a description and getting a 3D object could speed up surgical planning, custom implants, and teaching models. General generators, though, are weak on medical shapes.

They fine-tuned OpenAI's open Shap-e model on a set of biomedical objects, calling the result Shap-MeD. On their test set its latent error fell to 0.089, against 0.147 for the base model. Side-by-side looks at generated objects also favored the tuned model on structural accuracy. The authors present it as an assistant that cuts modeling time for medical designers.

Catalog record

CREATORNicolas Laverde 1, Melissa Robles 2, Johan Rodríguez 3
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE37 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2503-15562

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Shap-MeD Nicolas Laverde 1, Melissa Robles 2, Johan Rodríguez 3 Abstract We present Shap-MeD, a text-to-3D object generative model specialized in the biomedical domain. The objective of this study is to develop an assistant that facilitates the 3D modeling of medical objects, thereby reducing development time. 3D modeling in medicine has various applications, including surgical procedure simulation and planning, the design of personalized prosthetic implants, medical education, the creation of anatomical models, and the development of research prototypes. To achieve this, we leverage Shap-e, an open-source text-to-3D generative model developed by OpenAI, and fine-tune it using a dataset of biomedical objects. Our model achieved a mean squared error (MSE) of 0.089 in latent generation on the evaluation set, compared to Shap-e’s MSE of 0.147. Additionally, we conducted a qualitative evaluation, comparing our model with others in the generation of biomedical objects. Our results indicate that Shap-MeD demonstrates higher structural accuracy in biomedical object generation. I Introduction The new trend in healthcare is personalized treatments, moving away from universal approaches and one-size-fits-all treatments. According to the National Health Service (NHS) of England, universal treatments exhibit a 70% inefficiency rate among patients, highlighting the need for personalized treatments [ 1 ] . However, implementing a personalized treatment approach is unfeasible with traditional manufacturing processes, as it involves time- and labor-intensive tasks. In contrast, 3D-printed products have been well received by patients [ 2 ] . This trend underscores the necessity for the industry to transition toward new technologies that enable personalization. 3D printing was introduced more than three decades ago. Since then, this technology has radically transformed the manufacturing industry. The original purpose of this technology was the creation of engineering prototypes, including automotive parts, fashion accessories, and even architectural models, which were produced using this method. In fact, it is still used for these purposes today [ 3 , 4 , 5 ] . However, its applications extend far beyond these initial uses. 3D printing has undoubtedly revolutionized the healthcare system, as it enables the production of customized objects, making it particularly suitable for the personalization of prosthetics, implants, drug delivery systems, tissues, and medical devices [ 6 ] . Figure 1 illustrates the five major categories in which 3D printing is employed. One of the most pressing issues in healthcare that 3D printing addresses is the shortage of organs for transplantation. According to organ donation statistics, approximately 20 patients die each day while waiting for a transplant [ 7 ] . In addition to the limited availability of organ donors, another significant challenge is ensuring the biocompatibility of the donated organ [ 8 ] . 3D printing has the potential to generate tissues with patient-specific dimensions and materials, which could significantly contribute to organ transplantation efforts [ 9 ] . Figure 1: Current applications of 3D printing in medicine and healthcare. [ 10 ] Significant advancements have been made in 3D printing for personalized treatments. Structures can be generated from a digital 3D file using computer-aided design (CAD) software and computer vision techniques, including magnetic resonance imaging (MRI) or 3D tomography [ 11 ] . Although the results are highly promising, it is important to note that the level of personalization achievable with computer vision techniques is limited by the quality of the acquired images. The objective of this project is to develop an assistant dedicated to the creation of anatomical models and organs, conditioned by text or images. This aims to facilitate the development of research prototypes for tissue engineering and explore their potential applicability in prosthetic transplants. Additionally, the system is envisioned as an educational tool, benefiting students by enhancing their learning and understanding of anatomy and medicine. In line with the concept of treatment personalization, we aim to incorporate patient-specific information into the design of these 3D representations. This will be achieved through fine-tuning generative models based on diffusion models using biomedical objects. While strategies for generating conditioned 3D meshes will be presented later, it is important to emphasize that none of the existing approaches focus specifically on the biomedical domain. II Background Determining the methodology for representing three-dimensional objects presents a challenge due to the inherent complexity of shapes, variations in lighting, and spatial dimensions. Various strategies are employed to address different aspects of 3D representation, including the use of polygonal meshes, point clouds, and implicit function-based approaches. Examples of these techniques include the STL, XYZ, and NeRF formats, which correspond to each of the aforementioned methods, respectively. This section describes the different representations necessary to understand the models used throughout the project. II-A STL The three-dimensional representation of an object using an STL file involves a geometric description through polygonal meshes. This file format encodes the object’s surface as a network of triangles, where each vertex defines a point in three-dimensional space. An STL representation is thus a matrix of dimensions (T,3,3) , where T represents the number of triangles in the model, and each triangle is represented by a 3\times 3 matrix of the form \displaystyle\begin{bmatrix}x_{1}&y_{1}&z_{1}\\ x_{2}&y_{2}&z_{2}\\ x_{3}&y_{3}&z_{3}\\ \end{bmatrix}. Additionally, the components of the unit normal vector to each triangle are included, ensuring that it points outward according to the model’s orientation. This representation enables the transfer of 3D models between different programs and systems, facilitating their use in applications such as design, engineering, and manufacturing. II-B OBJ The OBJ format, also known as the Wavefront OBJ file, is a simple representation for 3D objects, originally developed for the visualizer The Advanced Visualizer by Wavefront Technologies [ 12 ] . This format consists of a plain text file that summarizes the object’s information, including texture and color details. Due to these features, the OBJ format has gained greater popularity in 3D printing compared to the STL format. This format allows for the representation of more complex surfaces beyond a simple arrangement of triangles. A set V of vertices v_{1},v_{2},\dots,v_{n}\in\mathbb{R}^{3} is defined, and subsets of these vertices are specified to form the corresponding surfaces or hyperplanes. Additionally, the text file includes information related to the textures and colors of the defined surfaces. Unlike the STL format, this file type not only significantly enhances 3D representation by offering greater flexibility in defining hyperplanes but also enables the inclusion of Freeform curves and surfaces. These curves are constructed using polynomial interpolation between a finite set of vertices, resulting in the approximation of smooth and continuous 3D surfaces that conform to the selected vertices, thereby generating a more seamless structure. II-C NeRF The NeRF 3D representation was proposed in 2022 [ 13 ] as an alternative to the classical representations used until then. It is based on the construction of an implicit function that maps tuples (\mathbf{x},\mathbf{d}) of coordinates and directions to tuples (\mathbf{c},\sigma) of colors and densities. The input coordinates, denoted as \mathbf{x}=(x,y,z)\in\mathbb{R}^{3} , are represented as points, while the directions are expressed using two angles, (\theta,\phi)\in\mathbb{R}^{2} , which define the viewpoint from which the object is observed. On the other hand, colors follow the RGB representation (c_{R},c_{G},c_{B}) , and densities are real numbers that symbolize opacity or the amount of matter at a given location within the three-dimensional volume of the scene. In summary, the function can be characterized as \displaystyle F_{\Theta}:\mathbb{R}^{5} \displaystyle\rightarrow\mathbb{R}^{4} \displaystyle(\textbf{x},\textbf{d}) \displaystyle\mapsto(\textbf{c},\sigma). The

Read the complete text