mitransient : Transient light transport in Mitsuba 3
Diego Royo, Jorge Garcia-Pueyo, Miguel Crespo, Oscar Pueyo-Ciutad, Guillermo Enguita, Diego Bielsa · 2025
Read mitransient : Transient light transport in Mitsuba 3 by Diego Royo, Jorge Garcia-Pueyo, Miguel Crespo, Oscar Pueyo-Ciutad, Guillermo Enguita, Diego Bielsa (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-25660.txt.
Introduction
Diego Royo led five co-authors presenting mitransient. Jorge Garcia-Pueyo, Miguel Crespo, and Guillermo Enguita helped. Oscar Pueyo-Ciutad and Diego Bielsa complete six authors. Their 2025 paper extends the Mitsuba 3 renderer. Transient light transport adds time-resolved run capability. Conventional rendering lacks a time dimension for light.
Mitransient accounts for time of flight of light. Novel transient imaging systems can be prototyped rapidly. Costly or difficult hardware is not needed first. Python modules run on CPU and GPU hardware. Just-in-time capabilities of Mitsuba 3 enable that. Physically-based runs include realistic materials and media. Participating media like fog or smoke are supported. Time-resolved polarization tracking extends the functionality further. Transient differentiable rendering is also supported now. Tools simplify non-line-of-sight imaging runs greatly. Realistic scene setups with capture noise simulate quickly. The community gains support for transient imaging methods.
Catalog record
| CREATOR | Diego Royo, Jorge Garcia-Pueyo, Miguel Crespo, Oscar Pueyo-Ciutad, Guillermo Enguita, Diego Bielsa |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 14 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2510-25660
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
mitransient : Transient light transport in Mitsuba 3 Diego Royo, Jorge Garcia-Pueyo, Miguel Crespo, Oscar Pueyo-Ciutad, Guillermo Enguita, Diego Bielsa Abstract . mitransient is a light transport simulation tool that extends Mitsuba 3 with support for time-resolved simulations. In essence, mitransient extends conventional rendering by adding a temporal dimension which accounts for the time of flight of light. This allows rapid prototyping of novel transient imaging systems without the need of costly or difficult-to-operate hardware. Our code is trivially easy to install through pip , and consists of Python modules that can run both in CPU and GPU by leveraging the JIT capabilities of Mitsuba 3. It provides physically-based simulations of complex phenomena, including a wide variety of realistic materials and participating media such as fog or smoke. In addition, we extend Mitsuba 3’s functionality to support time-resolved polarization tracking of light and transient differentiable rendering. Finally, we also include tools that simplify the use of our simulations for non-line-of-sight imaging, enabling realistic scene setups with capture noise to be simulated in just seconds of minutes. Altogether, we hope that mitransient will support the research community in developing novel algorithms for transient imaging. Figure 1 . mitransient can simulate a wide variety of time-resolved and time-gated capture devices, with support for complex materials, participating media, polarization tracking, differentiable rendering and non-line-of-sight imaging. Some examples: (a) Staircase scene simulated using transient_path integrator. From left to right: conventional camera, time-gated camera, peak time visualization, warped peak time visualization, and transient forward derivatives (with respect to the material of the floor). (b) Bottle scene filled with water and milk, being illuminated by a laser from its bottom. Up: real scene and frames of the propagation of light through the bottle captured with a streak camera. Bottom: synthetic scene and frames computed by our transient_volpath . 1. Introduction Transient imaging aims to capture and analyze how light propagates and interacts through a scene with an ultra-high temporal resolution. Thanks to recent advances in hardware, current technology can reach resolutions in the order of picoseconds. This increase of information about the temporal domain provides several benefits that have led to numerous applications, such as visualization of light in motion ( Jarabo et al. 2014 ) (Fig. 1 ), vision through highly-scattering media ( Luesia et al. 2022 ) , and inferring material properties from a distance. One of the most prominent applications of transient imaging is the reconstruction of hidden objects from captured indirect illumination (Fig. 3 ). This problem, known as non-line-of-sight (NLOS) imaging, is especially relevant to our work. Unfortunately, most methods that rely on transient imaging require expensive hardware, including ultra-fast cameras and pulsed illumination devices, which are difficult to calibrate and operate. Transient light transport simulation emerges as an alternative tool for developing, prototyping, and testing such systems, without hardware difficulties ( Jarabo et al. 2014 ) . However, there are no available state-of-the-art systems that fulfill the needs of the users, meaning that they are ad-hoc implementations for specific tasks (difficult to extend to other applications) or they lack novel optimized strategies (they are not as fast as they could be). Our rendering tool, mitransient , fixes these issues by being able to simulate different sorts of time-resolved sensing devices (including time-gated and transient cameras), with easy-to-extend modules written in Python that run in both CPU and GPU by using vectorized JIT compiled code from Mitsuba 3 ( Jakob et al. 2022 ) . This article showcases a few of the capabilities of mitransient : • Complex materials and participating media (Sec. 2 ). • Tracking the polarization of light in time, including a suite of polarization visualization tools (Sec. 3 ). • Inverse/differentiable rendering in the time domain. This allows you to compute gradients of transient renders with respect to scene parameters, enabling applications such as inverse rendering in the transient domain (Sec. 4 ). • Simulation of non-line-of-sight (NLOS) setups , allowing to simulate the capture process of illuminating and capturing points in the relay wall (Sec. 5 ). mitransient can be installed from PyPi 1 1 1 https://pypi.org/project/mitransient/ . Documentation is in Readthedocs 2 2 2 https://mitransient.readthedocs.io , and issues/pull requests are available on GitHub 3 3 3 https://github.com/diegoroyo/mitransient/ . 2. Transient rendering Our additions to Mitsuba 3 include extensions to steady-state path-tracing algorithms following the theory of the work of Jarabo et al. ( Jarabo et al. 2014 ) , including both surface and volumetric transport, referred in our code as the transient_path and transient_volpath integrators. Notice that our system is capable of simulating conventional, time-gated, and transient cameras, including effects such as time unwarping ( Velten et al. 2013 ) . Furthermore, both RGB and spectral rendering are supported. Fig. 1 shows multiple results from our transient algorithms in both transport regimes: (a) We leverage the complex Staircase scene to demonstrate several computations that can be done with our system with light having multiple bounces inside of it. In addition, (b) presents a recreation of the iconic Bottle scene from Velten et al. ( Velten et al. 2013 ) , including a bottle containing water with some drops of milk for added scattering, which we model using a rough plastic material containing a participating medium with coefficients similar to the physical ones. Rendering your first scene with mitransient is easy. The code below will render the famous Cornell Box scene in the transient domain and show a video of the results. 1: import mitsuba as mi 2: mi.set_variant( ’llvm_ad_rgb’ ) 3: import mitransient as mitr 4: 5: scene = mi.load_dict(mitr.cornell_box()) 6: img_steady, img_transient = mi.render(scene, spp= 1024 ) 7: 8: img_transient = mitr.vis.tonemap_transient(img_transient) 9: mitr.vis.show_video(img_transient) 3. Transient polarization tracking mitransient supports tracking polarization through time, capturing the polarization state of incoming light at each time bin, represented by a 4\times 1 Stokes vector (per color channel): (1) \mathbf{S}(t)=\begin{bmatrix}S_{0}(t)\\[3.0pt] S_{1}(t)\\[3.0pt] S_{2}(t)\\[3.0pt] S_{3}(t)\end{bmatrix},\quad\begin{array}[]{@{}l@{}}=\text{total intensity of the light}\\[3.0pt] =\text{linear polarization (horizontal\textendash vertical)}\\[3.0pt] =\text{linear polarization (diagonal)}\\[3.0pt] =\text{circular polarization}\end{array} Rendering polarization in transient state is akin to rendering polarization in steady-state rendering ( Wilkie and Weidlich 2012 ) . The throughput value in polarized rendering is a Mueller matrix \boldsymbol{\beta} that tracks the polarization interaction each scattering event in each path vertex. Following Mitsuba3 convention, each interaction is computed in local coordinates and transformed into its global coordinates through the si.to_world_mueller() function. Therefore, as we are tracking radiance in transient path tracing, the path’s throughput is updated as \boldsymbol{\beta}=\boldsymbol{\beta}\;\boldsymbol{M}_{i} , where \boldsymbol{M}_{i} is the Mueller matrix of the scattering event at the i -th path vertex. Figure 2 . Rendering polarization in transient rendering. We support in mitransient rendering the polarization state of transient light transport, by capturing the Stokes vector of radiance reaching the sensor. In the Spaceship scene, the polarization state at different frames varies due to the propagation delays between interactions of light in the spaceship. The angle of linear uses a rainbow colormap and the degree polarization a scale of reds, following the convention on polarized rendering ( Wilkie and Weidlich 2010 ) . Fig. 2 shows an example in the Spaceship scene, showing the angle of linear polarization \psi=\text{arctan}2(S_{2},S_{1}) and degree of polarization D=\sqrt{S_{1}^{2}+S_{2}^{2}+S_{3}^{2}}/S_{0} over time. 4. Differentiable transient rendering Differentiable rendering allows to track