RASP: Revisiting 3D Anamorphic Art for Shadow-Guided Packing of Irregular Objects Soumyaratna Debnath, Ashish Tiwari, Kaustubh Sadekar, Shanmuganathan Raman Abstract Recent advancements in learning-based methods have opened new avenues for exploring and interpreting art forms, such as shadow art, origami, and sketch art, through computational models. One notable visual art form is 3D Anamorphic Art in which an ensemble of arbitrarily shaped 3D objects creates a realistic and meaningful expression when observed from a particular viewpoint and loses its coherence over the other viewpoints. In this work, we build on insights from 3D Anamorphic Art to perform 3D object arrangement. We introduce RASP, a differentiable-rendering-based framework to arrange arbitrarily shaped 3D objects within a bounded volume via shadow (or silhouette)-guided optimization with an aim of minimal inter-object spacing and near-maximal occupancy. Furthermore, we propose a novel SDF-based formulation to handle inter-object intersection and container extrusion. We demonstrate that RASP can be extended to part assembly alongside object packing considering 3D objects to be “parts” of another 3D object. Finally, we present artistic illustrations of multi-view anamorphic art, achieving meaningful expressions from multiple viewpoints within a single ensemble. Figure 1 : (a) An ensemble of arbitrary shapes casting the shadow of alphabets CVPR and numbers 2025, (b) a set of irregular objects packed inside a face-shaped container, (c) an assembly of parts of a vessel obtained through multi-view shadow guidance via RASP. 1 Introduction For centuries, artists have used their expressions to redefine the boundaries of visual art, demonstrating how art shapes reality and influences human perception. Their work has also broadly impacted technology, design, and engineering. In this work, we explore using a unique visual art form, 3D Anamorphic Art, to tackle challenges in irregular object packing and extend this approach to part assembly. Interestingly, we are not alone in using (and, in fact, developing) artistic expressions to address applications in computer vision. For example, DeepDream Mordvintsev et al. 2015 uses GAN-based style transfer to apply the style of classical paintings, like those of Van Gogh or Picasso, to contemporary images. RePaint Shi et al. 2018 leverages deep learning and 3D printing to replicate the colors and textures of paintings by optimizing ink layering. ScribGen Debnath et al. 2024 creates human-like scribbles and hand-strokes in different styles. Beyond images, researchers have expanded artistic techniques into 3D analysis and understanding, as in Shadow Art Mitra and Pauly 2009 , which uses shadows for 3D reconstruction, sketch-based 3D generation Sadekar et al. 2022 , expressive hand movements Gangopadhyay et al. 2023 , and the design of knot configurations for realistic visual effects Gangopadhyay et al. 2024 . In this work, we draw upon insights from 3D Anamorphic Art to address an important problem of packing arbitrarily shaped objects within a 3D bounding volume (or container). Packing has numerous practical applications, spanning fields such as combinatorial optimization Martello et al. 2000 ; Seiden 2002 , computational geometry Hu et al. 2020 ; Ma et al. 2018 , computer vision and graphics Liu et al. 2019 ; Chen et al. 2015 ; Saakes et al. 2013 , machine learning Hu et al. 2017 ; Zhao et al. 2021 , robotics Yang et al. 2021 ; Shome et al. 2019 ; Wang and Hauser 2020 , and logistics and manufacturing Zhao et al. 2023 . 3D Anamorphic Art. 3D Anamorphic Art involves arranging objects in 3D space such that the arrangement appears coherent and meaningful only from the specific viewpoint(s). In contrast, from any other view, it appears random. For example, artist Patrick Proško used this technique at the Illusion Art Museum Prague by arranging electrical appliances to form a portrait of Nikola Tesla Proško 2015 (Figure 2 (a)) and arranging musical instruments to create a likeness of the musician Bedřich Smetana Proško 2019 (Figure 2 (b)). This concept also extends to shadows cast by such arrangements under particular camera-light configurations. For instance, artists Tim Noble and Sue Webster created Dirty White Trash Nobel and Webster 1998 and Wild Mood Swings Nobel and Webster 2009 , where carefully arranged trash and wooden pieces cast shadows of people sitting beside each other (see Figure 2 (c, d)). Whether through perspective views from specific viewpoints or shadows created under particular camera-light configurations, these projections reveal valuable insights about the arrangement of objects in 3D space. Figure 2 : Illustration of 3D Anamorphic Art (a, b): Portrait of Nikola Tesla and Bedřich Smetana by Patrick Proško and Shadow Art (c, d): Dirty White Trash and Wild Mood Swing by Tim Nobel and Sue Webster. Relation with Object Packing. Consider a cuboidal bounding volume (or container), as depicted in Figure 3 . When fully packed, this cuboid would appear as a rectangle or square upon being viewed orthographically along any of its faces. Similarly, it would cast rectangular or square shadows under the same viewing configuration. Ideally, gaps between the objects would appear as holes in the shadow from at least one perspective. By considering these shadows (technically binary images or silhouettes of the packed state) as target projections, we aim to arrange arbitrarily shaped elements within a bounding volume, thereby tackling the packing problem. Interestingly, this approach also allows for bounding volumes of arbitrary shapes, where the target projections are simply the volume’s appearance from multiple viewpoints. If the elements to be arranged are whole 3D objects, this approach addresses the irregular object packing problem. However, if these elements are “parts” of another object altogether, it naturally extends to the part assembly problem. A key challenge in existing packing methods is managing overlaps or intersections among the elements to be packed. While some methods focus on simple geometries like cuboids Crainic et al. 2012 ; Yamazaki et al. 2000 , spheres Mackay 1962 , cylinders Stoyan and Chugay 2009 , or ellipsoids Kallrath 2017 , or rely on assumptions of convexity or concavity in polyhedrons Stoyan et al. 2005 ; Romanova et al. 2018 ; Liu et al. 2015 , often using simple heuristics Limper et al. 2018 ; Nöll and Strieker 2011 , others handle irregular shapes by employing voxelized representations Byholm et al. 2009 ; De Korte and Brouwers 2013 , which tend to be susceptible to approximation errors. This work introduces a signed-distance field (SDF) approach to manage inter-object intersection and container extrusion. Figure 3 : An example depicting the packed state of a cuboidal container and the associated silhouettes/shadows. Figure 4 : The proposed differentiable rendering-based pipeline for RASP. We use Adam optimizer with a learning rate of 1e-2 to 1e-4 over 1000 iterations. Packing the Kitchen dataset ( 106 objects, 80000 grid points) takes nearly 17 minutes, and reassembly for each object in the Fantastic Breaks dataset ( 2 parts, 80000 grid points) takes under 3 minutes. Contributions. In this work, we introduce RASP – R evisiting 3D A namorphic art for S hadow-guided P acking of Irregular Objects. Given a set of arbitrarily shaped objects, a bounding volume (or container), and information about the appearance of its projection/shadows from different viewpoints in the packed state: • We propose a differentiable rendering-based framework to tackle irregular object packing by drawing inspiration from 3D Anamorphic Art. Our goal is to achieve near-maximal occupancy and minimal inter-object spacing within a known bounding volume. • We present a novel SDF-based approach to manage inter-object intersections and object-container extrusions, enhanced by an image-based loss function. • We demonstrate that RASP can also be applied to part assembly without the need for explicit 3D ground truth supervision. Additionally, we illustrate compelling visual effects that cater to multi-view anamorphic art. • To the best of our knowledge, this is the first approach to address packing (and part assembly) using only shadows or projections, guided by the principles of differentiable rendering. Through this work, we push the boundaries of 3D anamorphic art to address an important research question: can shadows be used to automatically find an optimal arrangement of arbitrarily shaped 3D objects within a bounded volume? While our framework aims to provide practical solutions, it does not always yield an optimal arrangement due to the NP-hard nature of the packing problem Hartmanis 1982 . Nonetheless, this work introduces a new perspective on solving the 3D packing and arrangement problem using shadows as a guiding mechanism. 2 Related Works Object packing has been addressed using heuristic-based, model-based, learning-based, or policy-based (online) object packing. Several methods have assumed simple object geometries, such as cuboids Crainic et al. 2012 ; Yamazaki et al. 2000 , spheres Mackay 1962 , cylinders Stoyan and Chugay 2009 , or ellipsoids Kallrath 2017 , or leverage convexity and/or concavity in polyhedrons Stoyan et al. 2005 ; Romanova et al. 2018 ; Liu et al. 2015 . This section primarily discusses the methods dealing with irregular object packing. Heuristic-based Methods. Several previous attempts at object packing were primarily based on heuristic strategies. One method proposed by Wang et al. Wang et al. 2010 sequentially places objects employing the Deepest Bottom Left Fill (DBLF) heuristic strategy. However, it is shown to create holes, leading to empty spaces. Later, Wang and Hauser Wang and Hauser 2021 addressed this limitation by reducing unfilled gaps through height-map minimization, which in itself ignores the difference between positions at the same level. HAPE3D Liu et al. 2015 , another heuristic-based algorithm, deploys the principle of the minimum total potential energy for irregular polyhedrons. Even after allowing free translations and rotations, it fails to achieve high packing density. Lamas et al. Lamas-Fernandez et al. 2023 voxelized irregular objects and used meta-heuristic algorithms to optimize voxel representation that is limited due to memory constraints. Heuristic-based methods generally apply well to objects with only a few facets and struggle to achieve a high packing density while dealing with complex irregular objects. Learning-Based Methods. The key challenge for packing irregular objects is to handle object collision detection or intersection/overlap avoidance. Several different geometric representations of objects (say, objects tightly bound by a sphere or cuboid Yao et al. 2015 ) have been assumed by different methods for collision detection and packing optimization. Romanova et al. [1] pose the packing problem as a nonlinear optimization problem and propose quasi-phi functions for simple concave polyhedrons to describe non-overlapping and distance constraints. However, obtaining a local optimum takes longer, even with free translations and rotations. A few works Chen et al. 2015 ; Yao et al. 2015 learned to decompose the object into multiple small parts to perform packing to reduce the supporting materials, build time, and assembly costs of 3D printing. While Chen et al. 2015 used voxel-based representation, Yao et al. 2015 used level sets to represent the object parts. Figure 5 : (a) Target shadow a duck-shaped container, optimized arrangement with mesh intersections. Mesh enclosed within (b) sphere and (c) cube where the intersection region doesn’t include object meshes. (d) SDF of two non-intersecting and intersecting objects (intersecting points are marked in red), and SDF transformation within the container. Moreover, Zhao et al. Zhao et al. 2023 and Zhuang et al. Zhuang et al. 2024 divide the objects into convex shapes for efficient collision detection in physical simulation and pack the objects either using shaking of the container as per the dynamic principle or reinforcement learning in an online manner, respectively. Our work does not focus on online packing. Cui et al. Cui et al. 2023 performed efficient collision metric computation in the spectral domain using voxel representation and Fast Fourier Transform (FFT) for discrete placement search. They deploy a greedy strategy to sort objects by volume from largest to smallest and then pack sequentially to minimize the distance between the new and already placed objects at each placement. The closest to our setting is the work by Ma et al. Ma et al. 2018 that optimizes the orientation and position of the objects from their initial placement. However, their approach is multi-stage, involving combinatorial optimization that includes object swapping, enlargement, and replacement to reduce the gaps between the objects. Furthermore, they also minimize the height of a container to obtain an optimal one through binary search. Due to repeated processing, the method is heavily time-consuming. In contrast, given a set of irregular objects, ours is a single-stage end-to-end optimization pipeline that performs packing only via 2D shadow guidance without any intense calculation within the 3D bounding volume. While Ma et al. Ma et al. 2018 have the information about the object set and process only one object at a time, sequentially determining the initial placement, our proposed method considers a set of objects at a time. 3 Method 3.1 Overview The key idea of our work is to arrange a set \mathcal{S} of arbitrarily shaped 3D objects within an arbitrarily shaped container \mathcal{C} such that the resulting arrangement casts K different shadows (projections) when viewed from K different directions using a differentiable rendering-based optimization pipeline. Interestingly, the problem has two interpretations depending on how we consider the set \mathcal{S} . (a) 3D Packing problem : when \mathcal{S} contains arbitrarily shaped 3D objects and \mathcal{C} is the bounding container within the viewing volume. (b) Part Assembly problem : when \mathcal{S} contains “parts” of a 3D object (O) where this object O itself is treated as the container \mathcal{C} . In the case of packing, we might not always have a ground truth or a unique solution, i.e., objects can be arranged in multiple optimal ways to maximize occupancy and minimize spaces. However, we have a ground truth shape in a part assembly whose parts can be assembled only in one way. We demonstrate how this is implemented in Section 4 . Overall, we propose a differentiable-rendering-based framework that naturally caters to both these requirements, in addition to generating multi-view anamorphic art. Inspired by computational visual arts, some of the recent works based on shadow art Mitra and Pauly 2009 ; Gangopadhyay et al. 2023 ; Gangopadhyay et al. 2024 have demonstrated the potential of 3D shape understanding, analysis, and reconstruction through shadows. In this work, we take the first step towards leveraging the benefit of differentiable rendering and principles of visual arts Wu et al. 2022 to propose shadow-based 3D packing and part assembly. 3.2 Proposed Approach The flow of the proposed methodology has been outlined in Figure 4 . Consider a set of N arbitrarily shaped objects \mathcal{S}=\{{O}_{1},{O}_{2},…,{O}_{N}\} such that each {O}_{i} is represented as a triangular mesh, an arbitrarily shaped bounding container \mathcal{C} located inside the camera viewing volume \mathcal{V} , and shadow projection configuration \mathcal{X}=\{X_{k}=(I_{k},P_{k})|k=1,2,…,K\} . Here, \{I_{k}\}_{k=1}^{K} are the target images under the camera configuration \{P_{k}\}_{k=1}^{K} . Essentially, the camera configuration P_{k}=(R_{k},t_{k}) corresponds to the camera extrinsic parameters associated with image I_{k} . With \mathcal{S} , \mathcal{C} , and \mathcal{X} as input to the differentiable renderer \mathcal{R} , the objective here is to learn the optimal arrangement of objects in set \mathcal{S} inside the container \mathcal{C} placed within the viewing volume \mathcal{V} such that the resulting images \{\widehat{I_{k}}\}_{k=1}^{K} rendered under camera configurations \{P_{k}\}_{k=1}^{K} are close to the corresponding target images \{I_{k}\}_{k=1}^{K} . From the shadow’s perspective, the target images are silhouettes, and we assume that the light (to cast the shadows) will be co-located with the camera. Finding the arrangement of objects is equivalent to finding the rigid transformation, i.e. , rotation and translation \{R_{i},t_{i}\}_{i=1}^{N} of each object O_{i} in set \mathcal{S} . We use quaternions for obtaining rotations - (a) to avoid gimble lock through other representations like axis-angle rotation or rotation composition of rotation along x-y-z axes, (b) for smooth interpolation, and (c) to learn a smaller number of parameters compared to the rotation matrices. Specifically, we obtain the angle of rotations across each of the coordinate axes and convert them to quaternion representation for optimization. Figure 6 : Visualization of 3D Object packing into a cuboidal container using RASP over four different object categories from Zhao et al. 2023 . Each instance is labeled with the number of objects packed in the container and the packing density. Figure 7 : Qualitative comparison of object packing with RASP and Zhao et al. Zhao et al. 2023 for same set and same number of objects. Best viewed in PDF with zoom. 3.2.1 Objective Function We propose a combined loss function consisting of an SDF-based intersection loss, an image-based loss, and an object-container extrusion loss, as per Equation 1 . \centering\mathcal{L}_{total}=\mathcal{L}_{sil}+\mathcal{L}_{is}+\lambda\mathcal{L}_{ext}\@add@centering (1) Here, we set \lambda=0.001 Image-based (Silhouette) Loss. A straightforward approach is to optimize each object’s rotation and translation parameters based on image rendering loss, such as mean squared error, as per Equation 2 . \mathcal{L}_{sil}=\frac{1}{MNK}\sum_{k=1}^{K}\sum_{i=1}^{MN}||I_{k}(i)-\widehat{I_{k}}(i)||_{2}^{2} (2) Here, MN is the total number of pixels in the image. As shown in Figure 5 (a), the projection of the arrangement may appear feasible (apart from a few gaps where more objects could fit), but in 3D, objects can still intersect within a duck-shaped container. Figure 8 : (Top) Loss curves for packed configuration in Figure 6 (solid line) & Figure 10 (dashed line). (Bottom) Effect of different loss terms on the packing. Best viewed in pdf with zoom. Therefore, the loss functions should penalize mesh intersections. However, handling mesh intersections is complex; some methods Karras 2012 detect intersections by evaluating each triangular face of one mesh against others, which is computationally costly and impractical for large numbers of meshes. To simplify intersection computation, some approaches enclose objects within unit spheres or cubes, finding intersections among these simpler bounding shapes Yao et al. 2015 . Although faster, this approach often fails with asymmetrically shaped objects, as intersections between bounding spheres or cubes may not correspond to intersections among the actual object meshes, as shown in Figure 5 (b, c). This can lead to excessive penalization of intersections, causing the network to push objects apart and create unnecessary gaps. Intersection Loss. To overcome these challenges, we propose an SDF-based approach to handle both object-object intersections and object-container extrusion. The signed distance field (SDF) provides the shortest distance from a point to the surface, with a negative sign indicating the point is inside and a positive sign indicating it is outside the object. Specifically, we pre-compute the SDF of each mesh (using Wang et al. 2022 ) at fixed query points within the container \mathcal{C} (a few milliseconds on GPU) and deform the SDFs with every update of the learned rotation and translation parameters. Rather than recalculating the SDF for each rigid transformation, we warp the previous SDF using a linear transformation to match the new configuration, as shown in Figure 5 (d). The degree of intersection, thus, can be determined by identifying points where more than one object has a negative SDF value, as marked in red in Figure 5 (d). Figure 9 : Illustration of packing of 24 cubes, each of dimensions 1\times 1\times 1 cubic units, into a container of dimensions 6\times 6\times 6 cubic units, with varying silhouette widths to simulate physically correct packing for N1 indicates that there is an intersection of at least two objects at the point, and D_{is}(\mathbf{p})=0 represents that the point \mathbf{p} is not inside any object. Considering the set of query points \mathcal{C}_{p} inside the container \mathcal{C} , the intersection loss is defined as per Equation 4 . \centering\mathcal{L}_{is}=\sum_{\mathbf{p}\in\mathcal{C}_{p}}D_{is}(\mathbf{p})\@add@centering (4) Container Extrusion Loss. Let V_{i} be the set of vertices of the triangular mesh corresponding to the object O_{i} and S_{\mathcal{C}} be the SDF of the container \mathcal{C} . We define the container extrusion loss as per Equation 5 \centering\mathcal{L}_{ext}=\sum_{i=1}^{N}\sum_{\mathbf{v}\in V_{i}}\text{max}(-\epsilon,S_{\mathcal{C}}(\mathbf{v}))\@add@centering (5) Here, \epsilon , the distance between a vertex and the container’s boundary, creates a “buffer zone” around the container boundary to control when the extrusion loss begins to be active. While a higher value of \epsilon hinders the movement of objects inside the container, a very small value leads to portions of objects suddenly extruding out of the container boundary, leading to an unstable optimization. We find \epsilon=0.01 to be a reasonable value that avoids penalizing objects that are slightly inside the container but still close to the boundary. We observed that in some cases, silhouette loss was just enough to contain the objects with the container, especially for simple shapes. However, for more irregular shapes with asymmetric and skewed spread, extrusion loss proved to be helpful. Figure 10 : Illustration of packing arbitrarily shaped objects into arbitrarily shaped containers. The initial containers, the set of object shapes to be packed, and the final packed configuration are shown in the figure. 4 Experiments In this section, we showcase the results obtained through the RASP framework for irregular object packing drawing parallels with 3D Anamorphic art. Furthermore, we extend RASP to applications like part assembly and creating artistic illustrations. 4.1 Irregular Object Packing For 3D object packing, we demonstrate the results majorly on the objects from the IR-BPP dataset Zhao et al. 2023 , which contains objects from four categories: General , Kitchen , ABC , and Block Out . A few other illustrations also include objects from the VOLMAP dataset Cherchi and Livesu 2023 and SilNet dataset Zisserman and Wiles 2017 . Following the existing literature of 3D object packing, we define the packing efficiency/density (\rho) , as per Equation 6 . \centering\rho=\sum_{O_{i}\in\mathcal{C}}\frac{|O_{i}|}{|\mathcal{C}|}\@add@centering (6) Here, |O_{i}| and |\mathcal{C}| are the volume of the i^{th} object and container volume, respectively. Figure 6 , compares the packing density of irregular 3D objects from different categories into a typical cuboidal container. Interestingly, RASP does not require any regularity in object shapes and can handle arbitrary shapes of different sizes, achieving reasonably good packing configurations. Figure 11 : Extending RASP to perform part reassembly of broken objects. The figure showcases the target silhouettes and broken parts reassembled using RASP. Deciding the number of objects that can be maximally packed within a container is crucial for optimal packing. To start with, we decide the initial number of objects as N_{init} based on the average volume of the objects and the container volume, such that N_{init}=\frac{|\mathcal{C}|}{|O|_{avg}} . Consider N_{max} as an unknown upper bound, i.e. , the maximum a physical container can accommodate. If N_{init}