Monocular Normal Estimation via Shading Sequence Estimation Zongrui Li 1 Xinhua Ma 1 Minghui Hu 1 Yunqing Zhao 2 Yingchen Yu 2 Qian Zheng 3,4 Chang Liu 5 Xudong Jiang 1 Song Bai 2 1 School of Electrical and Electronic Engineering, Nanyang Technological University 2 ByteDance 3 College of Computer Science and Technology, Zhejiang University 4 The State Key Lab of Brain-Machine Intelligence, Zhejiang University 5 MoE Key Laboratory of Interdisciplinary Research of Computation and Economics, Shanghai University of Finance and Economics https://github.com/LMozart/ICLR2026-RoSE.git Abstract Monocular normal estimation aims to estimate the normal map from a single RGB image of an object under arbitrary lights. Existing methods rely on deep models to directly predict normal maps. However, they often suffer from 3D misalignment : while the estimated normal maps may appear to have a correct appearance, the reconstructed surfaces often fail to align with the 3D geometry. We argue that this misalignment stems from the current paradigm: the model struggles to distinguish and reconstruct varying geometry represented in normal maps, as the differences in underlying geometry are reflected only through relatively subtle color variations. To address this issue, we propose a new paradigm that reformulates normal estimation as shading sequence estimation, where shading sequences are more sensitive to various geometry information. Building on this paradigm, we present RoSE , a method that leverages image-to-video generative models to predict shading sequences. The predicted shading sequences are then converted into normal maps by solving a simple ordinary least-squares problem. To enhance robustness and better handle complex objects, RoSE is trained on a synthetic dataset, MultiShade, with diverse shapes, materials, and light conditions. Experiments demonstrate that RoSE achieves state-of-the-art performance on both synthetic and real-world benchmark datasets for object-based monocular normal estimation. Figure 1: We present RoSE , a method using video generative model for monocular normal map estimation, built on a new paradigm that reformulates normal estimation as a shading sequence estimation task. Results on complex and diverse scenario show that RoSE reconstructs fine-grained geometric details and generalizes robustly to unseen datasets, achieving state-of-the-art performance in object-based monocular normal estimation on benchmark datasets. Figure 2: Illustration of 3D misalignment. The estimated normal maps of previous methods may appear to have an overall correct color distribution, yet the reconstructed surfaces often fail to align with the accurate geometry details, showing over-smooth results. Our estimated normal map has a higher 3D alignment compared to others. 1 Introduction Normal maps of surface encode 3D geometry by representing the orientation of the surface elements at each pixel, making them essential for a wide range of applications, including relighting ( 43 ; 40 ; 30 ; 35 ; 32 ) , 3D mesh reconstruction ( 53 ) , and modern gaming pipelines ( 49 ; 37 ) . In early methods, capturing accurate normal maps of an object requires specialized equipment and incurs high cost, motivating the development of more intelligent methods that can reliably infer them from casually captured RGB images of the object. Figure 3: Validation of sensitivity to geometry variations for different representations , including the proposed shading sequence (left) and the normal map (right), measured by average total variation (TV). TV is computed as the mean magnitude of the first-order image’s gradient in terms of different representations, where higher TV indicates stronger sensitivity to spatial geometric variation. To tackle this task, previous works ( 54 ; 21 ; 28 ; 31 ; 17 ; 2 ) directly estimate normal maps from a single RGB image using deep models. Despite achieving promising results, these methods often produce normal maps that appear to have a correct appearance but fail to remain consistent with the underlying 3D geometry. We refer to this limitation as “ 3D misalignment ” (see Fig. 2 ). We argue that this limitation arises from the current paradigm, where the model learns to recover geometry primarily by aligning with the color representation of normal maps. The deep model struggles to distinguish and reconstruct fine geometric details because normal maps represent geometry in a highly compact form, where geometry variations across different positions appear only as subtle color differences. This issue becomes particularly severe in monocular normal estimation, where geometric details in the input are inherently more ambiguous, thus difficult to recover. To reduce 3D misalignment, this work proposes a new paradigm for normal estimation by modifying the training target. The main idea is to adopt a representation that is more sensitive to geometry variation as the training target, thereby enhancing the network’s ability to distinguish and reconstruct geometric details. Guided by this intuition, we propose using a shading sequence , defined as the clamped dot product between the normal map and a set of canonical light directions, as the new training target. The idea of using a shading sequence is motivated by two key observations. First, shading sequences capture geometry variation through brightness variation, while excluding material influences, making them sensitive to only the geometric variations, as illustrated in Fig. 3 . Second, predicting shading sequence given canonical light directions is equivalent to predicting the normal map, as shading sequence can be losslessly converted to the normal map ( 55 ) using an Ordinary Least Square (OLS) solver. Based on the new paradigm, we propose RoSE , a method that R eformulating normal estimation as the S hading sequence E stimation based on the monocular input image. Specifically, as the shading sequence can be regarded as a video, we leverage recent image-to-video generative models ( 45 ; 6 ) to predict shading sequences. A key benefit of this design is that the rich lighting priors encoded in large-scale video models facilitate the generation of accurate shading variations. Once the shading sequence is obtained, we recover the normal map using an OLS solver ( 50 ) . In practice, to enhance robustness when handling objects with more complex materials and lights, we train our model on a collected dataset named MultiShade, enriched with more diverse materials and light conditions compared to previous datasets. Experimental results demonstrate that our method achieves superior performance compared to state-of-the-art methods. Overall, we summarize the main contributions of this paper as: • We introduce a new paradigm that reformulates the task of monocular normal estimation as shading sequence estimation. • Under the new paradigm, we propose RoSE, a monocular normal estimation method using an image-to-video generative model that predicts a shading sequence of an object under predefined canonical parallel lights and analytically derives normal maps from it. • We train RoSE on MultiShade, a synthetic dataset with diverse material and light conditions. Experiments show that our method achieves state-of-the-art performance on multiple datasets, especially on the widely-used real-world benchmark datasets ( i.e , DiLiGenT, LUCES). 2 Related Works Monocular Normal Estimation. Despite persistent research efforts in monocular normal estimation, achieving high accuracy remains significantly challenging due to the complexity of this task, which requires the prediction of an accurate normal map with highly limited input information. Early works ( 15 ; 14 ; 16 ; 47 ; 57 ; 4 ; 27 ; 28 ; 46 ) relied on handcrafted features, empirical priors, or conventional deep neural networks. However, these methods often suffer from limited generalization ability. Recent methods based on generative models ( 45 ; 17 ; 20 ) , physics-inspired deep networks ( 3 ) , and auto-regressive frameworks ( 53 ) have demonstrated improved generalization ability and the capacity to estimate relatively accurate normal maps. However, the estimated normal maps suffer from 3D misalignment, a problem that stems from the current paradigm where the model fails to capture the compact information in the normal maps. To address this, other works ( 43 ; 21 ) attempt to first generate more input images under a set of predefined canonical lights and subsequently estimate normals from these multi-light images. Yet, the accuracy of such methods is often degraded by artifacts in the generated input images, resulting in more severe 3D misalignment. In contrast, we propose a new paradigm that uses shading sequences, a representation that is sensitive to geometry, as the training target, and leverage video generative models to estimate them based on the input image, which achieves improved 3D alignment. Recent advances in video generation ( 39 ; 38 ; 56 ; 23 ) have significantly improved video synthesis quality. Methods such as ( 6 ; 13 ; 1 ; 33 ; 18 ) can generate high-fidelity videos by enforcing temporal consistency across frames through deep architectures such as temporal UNets ( 6 ; 18 ) and Transformers ( 13 ; 33 ) . In 3D generation, video diffusion models are used to facilitate cross-view consistency ( 45 ; 42 ; 11 ) to improve the quality of generated 3D models. In 3D estimation, recent work ( 5 ) employs a video diffusion model for normal estimation. They focus on predicting per-frame normals for an input video. In contrast, our work leverages the capability of video generative models to predict a shading sequence that follows a predefined light path consisting of multiple canonical parallel lights, using only a single input image. This enables accurate monocular normal estimation for objects with diverse shapes and materials. Shading Utilization. In photometric stereo methods, shadings are often used to explain the behavior of a deep model in normal map estimation. Previous studies ( 9 ; 48 ) show that learned features closely resemble shading sequences, motivating the use of shading sequence supervision to improve network performance ( 48 ) . Inspired by these findings, we reformulate monocular normal estimation as a shading sequence estimation task and train a video diffusion model with shading sequences as targets. 3 Methods 3.1 On Equivalence of Normal Estimation and Shading Sequence Estimation Shading map and shading sequence . In this paper, we define shading map ( 48 ) as: \mathbf{S}\triangleq\{\mathbf{s}_{\mathrm{p}}=\max(\mathbf{n}_{\mathrm{p}}\cdot\mathbf{l},0)|\mathrm{p}\in\mathcal{P}\}, (1) where \mathbf{n} is the normal map, and \mathbf{l} is the direction of parallel light, \max(.,0) is the nonlinear maximum operation that clamp the negative values, \mathcal{P} is the points that belong to the object. Shading maps remove the effects of surface reflectance and occlusion-induced cast shadows while preserving the geometry information and the attached shadow. A sequence of shading maps obtained under multiple canonical lights, defined as a shading sequence , \mathbf{S}^{s}\triangleq\{\mathbf{S}_{i}\mid i\in{1,\dots,f}\} , provides sensitive cues to the underlying 3D geometry. Normal map estimation . Given an observed image \mathbf{I} of an object captured under arbitrary lights, the goal of monocular normal estimation is to recover the normal map \mathbf{N}\triangleq\{\mathbf{n}_{\mathrm{p}}|\mathrm{p}\in\mathcal{P}\} . This requires learning a mapping: \Phi:\mathbf{I}\rightarrow\mathbf{N}. (2) Previous methods rely on deep models to learn a direct color mapping from a single RGB image to a normal map. This often produces a visually aligned appearance but inaccurate 3D geometry ( i.e , the 3D misalignment). A more recent line of work ( 21 ; 43 ) first generates a series of RGB images under simple light sources and then estimates normals from them. The main idea of these works is to augment the input with additional generated images that provide additional cues, thereby improving the prediction of normal maps. However, as the materials, lights, and geometry in the input image become more complex, the process of generating additional RGB images itself introduces substantial bias, ultimately leading to more pronounced 3D misalignment artifacts. Shading sequence estimation . The shading sequence under a set of predefined, non-coplanar, parallel lights (canonical lights) can be converted into a normal map. This enables us to shift the training target to shading sequence prediction with lights that vary along a predefined path, \mathbf{L}\triangleq\{\mathbf{l}_{i}\mid i\in{1,\dots,f}\} . \Phi_{S}:\mathbf{I}_{g}\rightarrow\mathbf{S}^{s}, (3) where \mathbf{I}_{g} denotes the grayscale input image. Then, the shading-to-normal estimation \mathbf{S}^{s}\rightarrow\mathbf{N} can be solved via Ordinary Least Squares (OLS) ( 50 ) : \displaystyle\mathbf{N}=\arg\min_{\mathbf{N}}\|\mathbf{N}^{\top}\mathbf{L}-\mathbf{S}^{s}\|^{2}=(\mathbf{L}^{\top}\mathbf{L})^{-1}\mathbf{L}^{\top}\mathbf{S}^{s}. (4) The solution is unique when \mathbf{L} is full rank. In practice, however, the \max(\cdot,0) operation causes a truncation effect, which makes the OLS estimate biased if OLS is applied directly to the shading sequence. To address this issue, we solve for the normal using only shadings with values greater than 0, treating them as valid equations in OLS. 3.2 Shading Sequence-based Training Target Reformulating monocular normal estimation as shading sequence estimation introduces additional flexibility in designing the training target, since different choices of \mathbf{L} yield different shading sequences. As long as each surface point is illuminated by at least three non-coplanar parallel light sources ( i.e , the lighting matrix \mathbf{L} is full rank in Eq. ( 4 )), normal maps can be recovered from the shading sequence without information loss. In our setup, this means that each surface point should correspond to at least three positive shading values. In this paper, we adopt a classic ring light setup from photometric stereo ( 58 ) , where canonical lights are uniformly placed on a latitude ring in the upper hemisphere of the object’s surface, each light oriented toward the surface center (see Fig. 4 ). Figure 4: Ring light setup . With an appropriate choice of the ring’s latitude ( 45^{\circ} in our setup), these lights collectively illuminate all surface points. The remaining question is: How many lights with distinct positions do we need, in particular the minimum number l_{\min} , to guarantee that every surface point is illuminated by at least three light sources with positive shading values? We address this in Lemma 1 . Lemma 1 . Define a surface point as illuminated if its shading value is positive, i.e , \max(0,\mathbf{S})>0 . Under a single parallel light, at least half of the upper hemisphere is illuminated. Therefore, n=2 lights are sufficient to ensure that every point on the hemisphere is illuminated once. To further guarantee that every point is illuminated by at least m=3 different lights, the minimum number of lights is l_{\min}=m\times n=6 , with the light directions uniformly distributed along the ring. In our experiments, we observed that slightly increasing the number of light sources improved the accuracy of both normal and shading estimation. The best performance was achieved with 9 light sources, yielding a 0.74^{\circ} improvement on LUCES 36 compared to 6 lights. While appropriately introducing additional light sources can further enhance accuracy, it also incurs longer training and convergence time as well as higher resource consumption. For instance, under the same settings, performance dropped by 1.31^{\circ} under 12 lights. Figure 5: Pipeline of RoSE. Given a monocular RGB image under arbitrary light, RoSE first converts it into a grayscale image, which is then used to generate the shading sequence via a video diffusion model. This generation is guided by two complementary feature representations extracted from a CLIP encoder and a VAE encoder. Finally, an ordinary least squares problem is solved analytically to estimate the normal map from the generated shading sequence. We train the video diffusion model while freezing the CLIP and the VAE encoder. 3.3 RoSE: a Monocular Normal Estimator based on Video Generative Model Architecture of RoSE is shown in Fig. 5 . Firstly, the shading generator \mathbf{g}_{\theta}(\cdot) is designed to take grayscale images \mathbf{I}_{g} as input, effectively eliminating redundant chromatic information that may distract the model from learning geometric cues. It produces grayscale shading sequences that follow a predefined light path, introducing structured patterns and temporal coherence well-suited to video generation models. In this paper, we implement \mathbf{g}_{\theta}(\cdot) using a standard video diffusion U-Net composed of multiple spatial and temporal transformer blocks ( 45 ; 6 ) . The grayscale input image \mathbf{I}_{g} is used as an additional condition to guide the denoising process during shading sequence generation. Specifically, following previous works ( 21 ; 45 ; 6 ) , we adopt a similar dual-branch conditioning strategy that combines global guidance from CLIP embedding and local guidance from VAE latent concatenation to reuse the pre-trained weights of the model. 1) CLIP embedding : We extract a global feature vector \mathbf{c}_{g} from the input image using a pretrained CLIP encoder. This semantic embedding is injected into the denoising U-Net via cross-attention, guiding the shading generation with object-level context. 2) VAE latent concatenation : to preserve spatial details, we encode the grayscale input \mathbf{I}_{G} with a pretrained VAE encoder \mathcal{E} and concatenate the resulting latent with the noisy latent z_{t} at each denoising step. Since \mathbf{I}_{G} is single-channel, we replicate it to three channels before feeding it into the VAE and CLIP encoders: \mathbf{I}_{g}^{\prime}=\operatorname{repeat}(\mathbf{I}_{g},B\times H\times W\rightarrow B\times H\times W\times 3) , where B is the batch size. By combining both conditioning techniques, the model better preserves the fine geometric structures of the input image during generation, which is particularly important for accurate normal estimation. Finally, the generated latents are decoded by the VAE decoder \mathcal{D} and averaged across channels to obtain the final grayscale shading sequence. Training. During training, we use the standard training objectives on latent space encoded by \mathcal{E} . The video generative model will learn to predict the noise given the noisy latent \mathbf{z}_{t} , \mathbf{z}_{0}=\mathcal{E}(\mathbf{S^{s}}) , where \mathbf{z}_{t}=\alpha_{t}\epsilon+\sigma_{t}\mathbf{z}_{0} . The diffusion loss follows calculation of \mathbf{z}_{0} -reparameterization ( 22 ) : \mathcal{L}_{\mathrm{diff}}=\mathbb{E}_{\mathbf{z}_{0},c,t}\left\|\mathbf{z}_{0}-\hat{\mathbf{z}}_{0}\right\|^{2},\hat{\mathbf{z}}_{0}=\frac{\mathbf{z}_{t}-\alpha_{t}\mathbf{g}_{\theta}\left(\mathbf{z}_{t}|c^{\prime},t\right)}{\sigma_{t}}. (5) where \hat{\mathbf{z}}_{0} the one-step denoised version of {\mathbf{z}}_{t} . Dataset Curation . To improve robustness when handling more complex materials and lights, we curate a dataset named MultiShade, featuring diverse shapes, materials, and light conditions to ensure robust generalization. MultiShade is built upon a list of pre-filtered 3D models (90K) curated from Objaverse ( 12 ; 21 ) , a widely adopted resource for 3D generation and reconstruction. For each object, we render observed images under three lighting setups: (1) parallel lights randomly placed around the object; (2) point lights with randomly sampled positions and intensities; and (3) environment lights using high-dynamic-range (HDR) maps sampled from a public collection of 780 real-world environments ( 19 ) . Each object is rendered from six distinct viewpoints (top, left, right, bottom, front, and one random view) to ensure comprehensive geometric coverage. To avoid lighting from the object’s backside in each view, we apply view-dependent transformations to keep light sources in the upper hemisphere relative to the view direction. During rendering, we implement material augmentation to the dataset by either retaining the object’s original texture or applying material augmentation. With a probability of 0.5 , an additional material is assigned from the MatSynth dataset ( 44 ) , which contains 5,657 high-quality PBR materials. More specifically, we assign a probability of 0.25 to sample materials from the metallic category, while 0.25 to extract materials from non-metallic categories such as plastic, wood, and fabric. This augmentation improves surface diversity and model robustness, especially for metallic materials. All images are rendered using Blender at a resolution of 576\times 576 following ( 45 ) , generating approximately 3 million image-normal pairs. Precomputed shading sequences under known canonical light sources are also provided. More details on rendering parameters, camera setup, and augmentation strategies are in the appendix. 4 Experiments 4.1 Experiment setup Datasets . We evaluate the proposed method on widely used benchmarks, including LUCES ( 36 ) for near-light monocular normal estimation, DiLiGenT ( 41 ) for parallel-light settings, and a curated test set of 100 unseen objects from the Objaverse dataset ( 12 ) rendered with diverse materials and light conditions. Baselines. We compare RoSE with 7 other monocular normal estimation methods, i.e , GeoWizard ( 17 ) , DSINE ( 3 ) , StableNormal ( 52 ) , Lotus-G & Lotus-D ( 20 ) , Neural LightRig ( 21 ) , and NiRNE ( 53 ) . Implementation details and evaluation metrics. All training experiments are conducted on 8 \times NVIDIA H100 GPUs with 80GB memory. The model is trained at a learning rate of 1\times e^{-5} , using AdamW as the optimizer. The diffusion architecture follows previous work ( 45 ) . More details can be found in the appendix. To assess the accuracy of predicted normal maps, following the common protocol in previous works ( 52 ; 53 ; 3 ; 20 ) , we use the mean angular error (MAE) as the evaluation metrics for all experiments. Table 1: Quantitative comparison in terms of MAE ( \downarrow ) of the normal map on DiLiGenT benchmark dataset. Highlighted numbers indicate the best and second best results among monocular estimation methods. † † nicematrix-placeholder: NiceTabular (nicematrix) Table 2: Quantitative comparison in terms of MAE of the normal map on LUCES benchmark dataset ( 36 ) . Highlighted numbers indicate the best and second best results among monocular estimation methods. † † nicematrix-placeholder: NiceTabular (nicematrix) 4.2 Performance on Benchmark Dataset We conduct monocular normal estimation experiments on the DiLiGenT ( 41 ) and LUCES benchmark dataset ( 36 ) to evaluate our method’s ability in handling objects captured under distant and near-field light sources. For each object, we select 10 images with relatively centered lights so that the light can cover enough details. The index of the images used for testing can be found in the appendix. Figure 6: Qualitative comparison on selected objects from two benchmark dataset ( Cow from DiLiGenT ( 41 ) and Squirrel from LUCES ( 36 ) . Row 1 & 3: normal map comparison. Row 2 & 4: error map comparison.) Best viewed in color with zooming in. Table 3: Quantitative comparison in terms of Mean and Median Angular Errors of the normal map on MultiShade test set, and the percentage of objects below a specific error bound. Highlighted numbers indicate the best and second best results among monocular estimation methods. † † nicematrix-placeholder: NiceTabular (nicematrix) Quantitative analysis on normal estimation . The results in Table 1 and Table 2 present the average MAE for each object across selected 10 images, and the average MAE across all objects. These quantitative results demonstrate a significant advantage of our method over the state-of-the-art method ( 16.36^{\circ} for ours vs. 17.27^{\circ} for NiRNE ( 53 ) on DiLiGenT dataset ( 41 ) ; 14.48^{\circ} for ours vs. 17.44^{\circ} for Lotus-G 20 on LUCES dataset ( 36 ) ). This validates the effectiveness of our method in achieving robust performance in normal estimation across various materials and light conditions. However, we observe that for certain objects, such as Goblet in DiLiGenT and House in LUCES, our method does not rank within the top two. We attribute this to the model’s inherent variance. Note that even the previous SOTA method, NiRNE, fails to deliver consistently strong performance across all cases. Another possible reason may be attributed to the training set used. We discuss this in Sec. 4.4 . Qualitative analysis on normal estimation . We present a qualitative comparison between our method and state-of-the-art methods in Fig. 6 . Our method consistently recovers accurate object details in the estimated normal maps, achieving lower MAE. In contrast, previous methods often produce overly smooth results, distorted normal distributions, or significant artifacts ( 21 ) ( e.g. , the tail and back of the Squirrel ). These results validate the effectiveness of the proposed shading sequence-based formulation and demonstrate RoSE’s advantage in preserving fine shape details for accurate normal estimation. Analysis on shading sequence estimation . We conduct quantitative analyses 1 1 1 Please refer to the appendix for qualitative analyses. of the predicted shading sequences on the LUCES dataset to illustrate RoSE’s ability to recover accurate shading sequences. The shading map of all other methods (including the ground truth) is computed as the dot product between the lights’ directions and the surface normal, with negative values clamped to zero. We use PSNR ( \uparrow ), SSIM ( \uparrow ), and LPIPS ( \downarrow ) as the evaluation metrics, as shown in Table 4 . The results demonstrate that RoSE achieves SOTA performance in predicted shading sequence, which also align with the results of normal estimation. Table 4: Quantitative comparison on estimated shading sequence in terms of PSNR ( \uparrow ), SSIM ( \uparrow ), and LPIPS ( \downarrow ) on LUCES benchmark dataset ( 36 ) . Highlighted numbers indicate the best and second best results. Metrics GeoWizard DSINE StableNormal Lotus-D Lotus-G Neural LightRig NiRNE Ours PSNR ( \uparrow ) 16.86 17.05 18.40 18.80 19.19 17.88 18.99 20.74 SSIM ( \uparrow ) 0.6920 0.7199 0.7411 0.7492 0.7589 0.7139 0.7503 0.7744 LPIPS ( \downarrow ) 0.2806 0.3100 0.2972 0.2868 0.2724 0.2831 0.2688 0.2583 4.3 Performance on MultiShade To further evaluate our method’s performance across various lighting conditions and materials, the test set of the applied synthetic dataset consists of 100 unseen objects from Objaverse ( 12 ) . Each object is rendered with random materials selected from the MatSynth test set ( 44 ) . For lighting conditions, we employ one random point light, one directional (parallel) light, and two environmental lights selected from Poly Haven ( 19 ) that are different from the training dataset. Each object is rendered from seven viewpoints, including the front, back, left, right, and top views, as well as two randomly sampled views. This setup yields a total of 2800 test samples. Following the evaluation protocol in prior work ( 21 ) , we report the mean and median angular error (MAE) across all objects, as well as the percentage of objects with MAE below specified angular thresholds. As shown in Table 3 , our method consistently outperforms baseline approaches across all metrics, with particularly strong performance under tighter thresholds ( i.e , 3^{\circ} - 7.5^{\circ} ), highlighting the robustness and accuracy of the proposed RoSE. 4.4 Ablation Study We conduct ablation experiments using the LUCES benchmark dataset ( 36 ) as the test set to analyze the effectiveness of the proposed RoSE and MultiShade. Additional experiments, analysis, and discussion are in the appendix. Validation on details alignment. Following ( 53 ) , we evaluate detailed alignment by computing the sharp normal error (SNE), i.e , the normal estimation error measured in boundary regions, on the estimated normal maps from the LUCES dataset, see Table 5 . Our method achieves performance comparable to the state of the art method (NiRNE 53 ) and shows a clear advantage over other baselines. It is worth noting that NiRNE was trained on a dataset nearly 10\times larger and containing significantly more diverse and complex 3D models than ours. These results highlight that the proposed RoSE is capable of generating fine-grained details even with substantially lower resource consumption during training. Validation on negative-clamping on shading sequence. After clamping negative values, the shading sequence is rescaled to the range [-1,1] (by applying a linear transformation \mathbf{S}\mapsto\mathbf{S}\times 2-1 ) to match the input requirements of the VAE encoder 6 . This rescaling makes the shading sequence more sensitive to geometric variations. The effectiveness of this strategy is validated in Table 6 with comparison between ‘ours’ and ‘ours w/o clamp’. Table 5: Quantitative analysis in terms of MAE and SNE of the normal map on LUCES benchmark dataset ( 36 ) . Highlighted numbers indicate the best and second best results. † † nicematrix-placeholder: NiceTabular (nicematrix) Table 6: Ablation study in terms of MAE of the normal map on LUCES benchmark dataset ( 36 ) . In particular, “+M”(“+L”) means training on Multishade (LightProp) dataset, ‘w/o clamp’ means removing clamping on shading sequence. ‘w/o MA’ means training on dataset without material augmentation. Highlighted numbers indicate the best and second best results. † † nicematrix-placeholder: NiceTabular (nicematrix) Validation on material augmentation . To evaluate the effectiveness of the proposed dataset, we train RoSE on the publicly available LightProp ( 21 ) dataset as a reference. The key difference is that our dataset introduces additional material augmentation to increase material diversity. As shown in Table 6 , training without material augmentation (w/o MA, using only the original object materials) yields slightly worse performance on the LUCES ( 36 ) benchmark, while enabling material augmentation leads to notable improvements. These results demonstrate the effectiveness of the material augmentation. Validation on dataset impact . We retrain the previous SOTA method on LUCES ( i.e , Lotus-G) using our dataset (“Lotus-G+M”), resulting in consistent improvements and further validating the effectiveness of the proposed dataset. More importantly, when trained on the same dataset, our method still outperforms the baselines: Neural LightRig vs. “Ours+L” and “Lotus-G+M” vs. Ours, clearly demonstrating SOTA performance and highlighting its efficiency and competitiveness. Finally, we also observed that for some specific objects, such as House , retraining Lotus-G with our dataset resulted in decreased performance (35.32 ∘ for Lotus-G vs. 38.90 ∘ for “Lotus-G+M”). This suggests that dataset variations may affect estimation accuracy on certain objects. Validation on model variants impact . We train RoSE using a different video diffusion backbone, namely Stable Video Diffusion XL (SVD XL) ( 6 ) , on the MultiShade dataset. We denote this variant as “Ours w/ SVD XL”. As shown in Table 6 , this model achieves performance comparable to RoSE built on SV3D (14.58 ∘ for “Ours w/ SVD XL” and 14.48 ∘ for Ours). This demonstrates that our framework generalises well even when the backbone is pretrained on large-scale, general-purpose video data rather than a domain-specific object-centric dataset. Validation on gray-scale input . We train a variant of RoSE that replaces the grayscale input with an RGB input ( i.e , Ours w/ RGB). The performance drops by 0.79^{\circ} on the LUCES benchmark. The result indicates the effectiveness of the grayscale input, which eliminates redundant chromatic information to enable accurate shading sequence estimation. Validation on ring-light setup . We train RoSE using a different light path where the elevation decreases from 60^{\circ} to 30^{\circ} while rotating 360^{\circ} around the z -axis. We denote this variant as “Ours w/ spiral”. As shown in Table 6 , this more complex light path leads to a performance drop (MAE of 17.60^{\circ} ). This result highlights that the proposed ring-light setup is an effective and efficient design for predicting the shading sequence. 5 Discussion Conclusion . We propose RoSE, a novel method for monocular normal estimation that addresses the limitations of previous methods in their training paradigms to reduce 3D misalignment. By reformulating normal estimation to shading sequence estimation, RoSE facilitates normal estimation through an image-to-video generative model and a simple analytical solver. To further improve performance across more general scenarios, we train RoSE on MultiShade, a large-scale dataset with diverse materials and lighting conditions. Experiments show that RoSE outperforms state-of-the-art methods. Limitations & Future Work . While RoSE demonstrates strong performance in normal estimation across various settings, it has several limitations. First, employing video diffusion models for shading sequence generation introduces additional computational overhead, which may limit the applicability of the method in real-time scenarios. Second, RoSE may struggle under extreme lighting conditions, particularly when large regions of the object receive insufficient illumination, resulting in degraded shading quality and less reliable normal predictions in those areas. Third, RoSE fails to produce high-quality normal maps on transparent or semi-transparent objects, and extending support for such cases will be an important direction for future work. Finally, the current evaluation is primarily object-centric, with a focus on robustness to varying light sources and reflectance properties. Extending RoSE to scene-centric settings remains an important direction for future work. Acknowledgment . This work is supported by the Ministry of Education, Singapore, AcRF Tier 1 grant No. RG98/24, and ByteDance Inc. Appendix A LLM Usage Statement Large Language Models (LLMs) were only used in this research for writing optimization and grammar checking. No part of the theoretical contributions, experimental design, data analysis, or results was generated by LLMs. Appendix B Broader Impact The proposed method improves the accuracy of normal map estimation from a monocular image, with broad benefits across various applications. More precise geometry understanding can significantly improve downstream tasks such as 3D reconstruction, augmented reality, robotics, and digital content creation, enabling more immersive and interactive user experiences. Appendix C Additional Implementation Details Training details . As stated in the main paper, we build RoSE on top of SV3D ( 45 ) . During training, we initialize our model with pretrained SV3D weights (1.5B) and fine-tune the first convolutional layer as well as all parameters within the self-attention and cross-attention modules on the MultiShade dataset, which contains 90K objects (3.1M image–normal pairs), and evaluated on a validation set of 100 objects, each providing 100 image–normal pairs, result in 200M trainable parameters. Training is conducted for 80,000 steps with a learning rate of 1\times 10^{-5} and a total batch size of 16. We use the AdamW optimizer with \beta_{1}=0.9 , \beta_{2}=0.999 , and 1\times 10^{-8} for weight decay. Training is performed in float16 precision for efficiency, and we apply gradient clipping with a maximum norm of 1.0. The model is trained to predict a 9-frame shading sequence, with each frame at a resolution of 576\times 576 . End-to-end training takes approximately one day on 8 NVIDIA H100 80GB GPUs. Testing details . We follow the requirements specified in the baselines’ inference code ( 21 ; 17 ; 52 ; 53 ; 3 ) to prepare our test dataset, ensuring compatibility with each setup for a fair comparison. For both the LUCES ( 36 ) and DiLiGenT ( 41 ) datasets, we use images indexed from 21 to 30 for testing, as the lights are more centered on the objects. All testing processes are performed on a single RTX A6000 Ada GPU. The total runtime includes the video diffusion model inference with 25 denoising steps and the shading to normal computation, the latter adding only a negligible cost of 0.045 seconds per object. For completeness, we also report the inference time of other methods for reference. Table 7: Average inference time of monocular normal estimation methods per image (in seconds). Method GeoWizard DSINE StableNormal Lotus G Lotus D Neural LightRig NiRNE Ours Time 101.11 0.83 1.52 0.61 0.59 93.73 0.31 10.57 Appendix D Additional Details about MultiShade More Details about Material Augmentation . We present a statistical comparison of the proposed dataset with other related datasets ( 21 ; 53 ; 26 ; 24 ; 25 ) in Table 8 , including works ( 21 ; 53 ) that are either recently released or not yet publicly available. We apply material augmentation (MA) with a probability of 0.5 by randomly replacing an object’s material with one sampled from the MatSynth dataset ( 44 ) , selecting equally from metallic (617) or non-metallic (5,040) material groups. This process yields an additional 42,732 objects that share the same 3D geometry but differ in material appearance. The resulting MultiShade dataset, enriched with material diversity and rendered shading sequences, enables our method to achieve state-of-the-art performance on public benchmarks. Table 8: Statistics of representative datasets used for normal estimation under arbitrary lighting. # O and # v denote the number of 3D models and rendered views, respectively. N.A. indicates that the corresponding information is not available. ‘env.’, ‘par.’, ‘poi.’ stand for environment light, parallel lights, and point lights, respectively. † † nicematrix-placeholder: NiceTabular (nicematrix) Rendering setup . We construct our dataset using the Cycles rendering engine in Blender ( 10 ) , selecting 90,546 filtered objects from Objaverse ( 26 ) . Each object is rendered from six viewpoints. For each view, we implement one parallel light, one point light, or two HDR environment maps, selected from a pool of 780 real-world HDR environments ( 19 ) . The directions of the point and parallel lights are randomly sampled from the upper-front hemisphere facing the camera (see Fig. 7 ). The camera is positioned at a random distance \tau between 1.5 and 1.8 meters from the object, with a focal length of 35 mm, following the setup in ( 34 ) . Figure 7: Image rendering setup. Appendix E Results on Web Images We present qualitative comparisons with state-of-the-art methods (NiRNE ( 53 ) and Neural LightRig ( 21 ) ) on additional images sourced from public resources, including the project page of StableNormal ( 52 ) and Google Images, as shown in Fig. 8 and Fig. 9 . The surface reconstruction from normals is performed using the method from ( 7 ) . Figure 8: Qualitative comparison of normal maps on web images. Figure 9: Qualitative comparison of normal maps on web images. Appendix F Additional Experiment Results on Popular Datasets F.1 Results on DiLiGenT We present a qualitative comparison of different methods for normal estimation. To avoid excessive redundancy, we select the normal map whose MAE is closest to the average MAE as a representative example for reference. Figure 10: Qualitative comparison on normal maps and error maps for the Goblet , Bear , Pot1 , Cat , Cow from the DiLiGenT ( 41 ) benchmark. Figure 11: Qualitative comparison on normal maps and error maps for the Ball , Harvest , Buddha , Pot2 , Reading from the DiLiGenT ( 41 ) benchmark. F.2 Results on LUCES We present a qualitative comparison on LUCES ( 36 ) of different methods for normal estimation. To avoid excessive redundancy, we select the normal map whose MAE is closest to the average MAE as a representative example for reference. Figure 12: Qualitative comparison on normal maps and error maps for the Owl , Queen , Squirrel , Tool from the LUCES ( 36 ) benchmark. Figure 13: Qualitative comparison on normal maps and error maps for the Cup , Die , Hippo , House , and Jar from the LUCES ( 36 ) benchmark. Figure 14: Qualitative comparison on normal maps and error maps for the Owl , Queen , Squirrel , Tool from the LUCES ( 36 ) benchmark. F.3 Results on LightProp We report quantitative results on the LightProp dataset ( 21 ) . Our method achieves the second-best overall performance in terms of mean and median angular errors for normal map estimation. We re-ran the entire evaluation, as we observed discrepancies between our results and those reported in the original paper ( 21 ) . The metric computation scripts used in our evaluation follow the implementation in ( 3 ) 2 2 2 https://github.com/baegwangbin/DSINE/blob/main/projects/dsine/test.py . Table 9: Quantitative comparison in terms of Mean and Median Angular Errors of the normal map on LightProp test set, and the percentage of objects below a specific error bound. Bold ( underline ) numbers indicate the best ( second-best ) results among single-view normal estimation methods. † † nicematrix-placeholder: NiceTabular (nicematrix) F.4 Results on Natural Light Photometric Stereo Dataset We conduct quantitative comparison results on a synthetic dataset (NaPS) proposed in ( 31 ) , which contains a series of rendered images based on objects selected from ( 41 ) , under varying environment lighting, material properties, and object shapes. The dataset is organized into four groups: (1) the light group, which evaluates performance on a single object under different lighting conditions; (2) the shape group, which compares different object geometries; (3) the reflectance group, which assesses performance on diffuse and specular materials; and (4) the spatially varying material group, which poses a challenging scenario with complex, spatially varying materials. During testing, we select the 10 images with the highest average brightness for evaluation. Our method achieves either the best or second-best performance in each group and ranks first in terms of overall average performance ( 11.35^{\circ} , ours vs. 12.32^{\circ} , NiRNE, the second best method). Table 10: Results on the NaPS ( 31 ) dataset for normal estimation under natural lighting. ‘L., A., S., U.’ denote four types of environment maps, Landscape, Attic, Studio, and Urban, covering both outdoor and indoor settings. ‘D., S.’ in the reflectance and spatially varying material groups refer to diffuse and specular materials, respectively. Numbers indicate the MAE ( \downarrow ) of the estimated normal maps. Bold ( underline ) numbers indicate the best ( second-best ) results. † † nicematrix-placeholder: NiceTabular (nicematrix) F.5 Overall Performance Comparison In this paper, we conduct a comprehensive analysis across multiple benchmark datasets, including two real-world datasets (DiLiGenT ( 41 ) and LUCES ( 36 ) ) and three synthetic datasets (MultiShade, NaPS ( 31 ) , and LightProp ( 21 ) ). According to Table 11 , our method achieves the best average performance in terms of MAE and overall ranking. This demonstrates the strong generalization ability of the proposed RoSE across diverse scenarios, lighting conditions, and object types. Table 11: Quantitative comparison over all five datasets. We report the MAE ( \downarrow ) over all objects and the rank ( \downarrow ) among methods at specific dataset for comparison. † † nicematrix-placeholder: NiceTabular (nicematrix) Appendix G Case Analysis We conduct a detailed analysis across variations in lighting, textures, and BRDFs through three groups of controlled experiments. The first group examines how different lighting conditions, including simple light, parallel light, point light, and challenging light, influence performance, with texture and BRDF fixed. The second group evaluates the impact of texture complexity by comparing a simple texture with three spatially varying textures, while holding lighting and BRDF constant. The third group isolates material effects by varying the BRDF under simple lighting and texture settings. The results (see Fig. 15 -Fig. 17 ) show that our method consistently produces high-quality normal maps relative to other approaches. However, performance does degrade under highly complex textures, challenging lighting conditions, or difficult material properties. Figure 15: Analysis of each method’s performance across different texture settings on Bunny . Figure 16: Analysis of each method’s performance across different BRDF settings on Bunny . Figure 17: Analysis of each method’s performance across different light settings on Bunny , where “Paral” indicate parallel lights, “Env” stands for environment light. Appendix H Additional Discussion H.1 Analysis on Latent’s Distribution We analyse the latent representations of RGB images, corresponding normal maps, and shading maps from the MultiShade after encoding them with the Stable Diffusion ( 39 ) VAE, in order to examine whether shading maps lie closer to RGB images than the normal map in latent space. Specifically, we randomly sample 1000 objects from the dataset. For each object, we randomly select one RGB image under a random viewpoint and lighting condition, retrieve its corresponding normal map, and randomly choose one shading map computed under 9 ring lights setup, giving us 3000 samples in total. Each sample has size 72 \times 72 \times 8. We average each latent across the 72 \times 72 spatial dimensions and apply t-SNE to obtain a 3D embedding for each sample. This results in three sets of 1000 \times 3 vectors, which we visualize in Fig. 18 . As shown, the shading latents exhibit a tendency to cluster closer to RGB image latents. Figure 18: Visualization of the distribution of the downsampled latents of normal maps, RGB images, and shading maps from MultiShade, shown from three orthogonally projected views. H.2 Discussion on Shading Sequence’s Robustness We conduct a 5-run experiment to evaluate how perturbations in the shading sequence affect the final normal estimation. In each run, identical Gaussian noise of varying magnitudes is added to one or multiple shading maps and the surface normal. The results on Bunny from LUCES ( 36 ) show that the shading sequence is noticeably robust to the perturbations. When noise is injected into a single shading map, the change in mean angular error over 5 runs remains small ( 1.537^{\circ} to 5.233^{\circ} ), and is substantially lower than perturbing the normal map directly ( 3.235^{\circ} to 24.421^{\circ} ), even under the strongest disturbance. A larger deviation emerges only when 9 shading maps are perturbed simultaneously ( 3.612^{\circ} to 20.713^{\circ} ), yet the shading-sequence formulation still remains more robust than directly perturbing normals. These observations confirm that predicting normals from shading sequences provides an inherent degree of robustness to noise. Table 12: Quantitative analysis of how perturbations to shading sequences and normal maps affect the mean angular error relative to the clean version. ‘x shading maps’ indicates that x frames in the shading sequence are perturbed by Gaussian noise. The first row specifies the noise magnitude. Perturbed Frames 0.050 0.100 0.200 0.300 0.400 1 Shading Map 1.537 2.405 3.479 4.452 5.233 5 Shading Maps 2.907 4.766 7.514 10.868 14.033 9 Shading Maps 3.612 5.745 10.468 15.727 20.713 Normal Map 3.235 6.454 12.764 18.825 24.421 H.3 Discussion on Different Shading-based Paradigm To compare the two paradigms, i.e , (1) first estimating normals and then computing a shading loss, and (2) our proposed paradigm that directly predicts the shading sequence and then derives normals, we evaluate the effect of incorporating a rendering-based shading loss into an image diffusion model. Specifically, we apply the shading loss to Lotus-G ( 20 ) , where the loss is defined as the L1 distance between the predicted shading sequence and the ground truth sequence. Each shading map is computed as the dot product between the predicted surface normal and the preset ring-light path, with negative values clamped to 0. This variant (“Lotus w/ shad.”) is trained on the MultiShade dataset. The results show that Lotus w/ shad achieves improved performance (16.20 ∘ MAE on the LUCES benchmark). However, it still falls short of our method (14.48 ∘ ). This supports the effectiveness of our framework: estimating the entire shading sequence with a video diffusion model allows the network to capture the interrelationships among shading maps that contain rich geometric information, thereby improving the quality of the estimated shading sequence and enabling more accurate normal estimation. Table 13: Comparison between “Lotus w/ shad.” and the proposed RoSE on the LUCES benchmark. Numbers indicate MAE ( \downarrow ) in degrees. Bold numbers indicate the best results. Method Ball Bell Bowl Buddha Bunny Cup Die Hippo House Jar Owl Queen Squirrel Tool Mean Lotus w/ shad. 19.65 7.66 9.76 14.95 9.43 20.45 12.43 11.00 36.54 13.06 23.16 18.01 18.07 12.64 16.20 Ours 9.09 5.94 6.84 17.58 12.70 13.80 8.26 14.14 36.79 5.93 19.60 19.99 21.34 10.66 14.48 H.4 Comparison with Multi-view Normal Estimation Methods We have conducted an additional comparison with multi-view normal estimation methods, including Era3D ( 29 ) and Unique3D 51 , results are shown in Table 14 . On the LUCES benchmark, Era3D and Unique3D achieve mean angular errors of 43.33° and 23.00 ∘ , respectively, both substantially worse than the proposed RoSE. Particularly, although Unique3D produces visually high-contrast normal maps, the underlying geometric details are inaccurate, and its performance degrades significantly on objects with complex shadow patterns, which is consistent with our observations of 3D misalignment. Table 14: Quantitative comparison with multi-view normal estimation methods on the LUCES benchmark. Bold number indicates the best results. Method Ball Bell Bowl Buddha Bunny Cup Die Hippo House Jar Owl Queen Squirrel Tool Mean Era3D 73.02 29.53 90.85 27.85 22.82 67.50 49.73 28.32 48.87 38.92 27.86 32.24 32.13 37.03 43.33 Unique3D 14.89 13.52 13.76 24.68 16.00 34.76 19.10 16.52 44.74 15.80 28.66 26.02 26.01 27.48 23.00 Ours 9.09 5.94 6.84 17.58 12.70 13.80 8.26 14.14 36.79 5.93 19.60 19.99 21.34 10.66 14.48 H.5 Analysis on 3D Reconstruction We further evaluate the quality of the reconstructed surfaces obtained from both the predicted and ground truth normal maps on the DiLiGenT and LUCES datasets, using the reconstruction method of ( 8 ) . The results are reported in Table 15 and Table 16 . By measuring the RMSE ( 8 ) between the reconstructed and groundtruth surfaces (obtained by reconstruction from the groundtruth normal map), we observe that our method consistently achieves state-of-the-art performance across both benchmarks. These results demonstrate that RoSE not only produces accurate normal maps but also preserves high-fidelity geometric structures after surface reconstruction, further validating the effectiveness of the proposed method. Table 15: Quantitative comparison of surface normal RMSE ( \downarrow ) on the DiLiGenT dataset. Results reported with values \times 10 . Bold number indicates the best results. Method Ball Bear Buddha Cat Cow Goblet Harvest Pot1 Pot2 Reading AVG Lotus-G 1.08 1.31 0.55 1.19 1.58 0.84 1.83 1.96 0.84 1.18 1.24 Lotus-D 2.37 0.63 0.52 1.02 1.30 0.76 2.25 1.91 1.01 1.02 1.28 Neural LightRig 1.07 0.88 1.51 2.69 1.35 1.26 5.75 2.40 0.91 1.38 1.92 NiRNE 1.46 0.94 0.48 0.77 1.47 2.12 1.54 0.98 0.88 1.63 1.23 Ours 0.79 0.88 0.33 0.92 0.65 1.16 1.24 1.35 0.75 1.33 0.95 Table 16: Quantitative comparison of surface normal RMSE ( \downarrow ) on the LUCES dataset. Results reported with values \times 10 . Bold number indicates the best results. Method Ball Bell Bowl Buddha Bunny Cup Die Hippo House Jar Owl Queen Squirrel Tool AVG Lotus-G 0.89 0.36 1.33 1.12 0.72 1.18 0.67 1.23 1.23 0.57 0.83 0.55 1.30 0.34 0.88 Lotus-D 0.89 0.38 1.09 1.13 0.73 1.82 0.75 1.09 1.54 0.56 0.89 0.46 0.86 0.31 0.89 Neural LightRig 0.31 0.51 3.02 0.57 0.69 0.98 1.71 1.51 1.28 1.88 1.05 0.45 0.60 0.65 1.09 NiRNE 0.77 0.76 2.12 0.70 0.64 1.10 1.02 1.71 1.12 0.75 1.93 0.75 0.71 0.71 1.05 Ours 0.34 0.19 0.80 0.69 0.53 0.72 0.54 0.79 0.67 0.42 1.22 0.25 0.54 0.22 0.57 H.6 Performance on Normal Estimation using Video Diffusion Model We have conducted an additional comparison with the variant that using SV3D to directly predict the single-frame surface normal (noted as “SVD-nml”), the results are shown in Table 17 . The average result on LUCES benchmark dataset is 20.61^{\circ} , indicating that the dense information geometrically encoded in the video model does not play a critical role in this setting, and the model also loses its ability to make use of temporal information. Table 17: Quantitative comparison with “SVD-nml” on the LUCES benchmark. Bold number indicates the best results. Method Ball Bell Bowl Buddha Bunny Cup Die Hippo House Jar Owl Queen Squirrel Tool Mean SVD-nml 17.20 9.96 19.69 22.28 14.77 18.32 11.47 18.23 49.74 10.49 26.95 25.04 26.03 18.29 20.61 Ours 9.09 5.94 6.84 17.58 12.70 13.80 8.26 14.14 36.79 5.93 19.60 19.99 21.34 10.66 14.48