ToF ReSTIR: Time-of-Flight Rendering with Spatio-temporal Reservoir Resampling 251 Journal: TOG Journal: TOG Volume: 45 4 89 7 DOI: 10.1145/3811299 CCS: Computing methodologies Ray tracing CCS: Computing methodologies Computational photography
Juhyeon Kim, Wojciech Jarosz, Adithya Pediredla · 2026
Read ToF ReSTIR: Time-of-Flight Rendering with Spatio-temporal Reservoir Resampling 251 Journal: TOG Journal: TOG Volume: 45 4 89 7 DOI: 10.1145/3811299 CCS: Computing methodologies Ray tracing CCS: Computing methodologies Computational photography by Juhyeon Kim, Wojciech Jarosz, Adithya Pediredla (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-11536.txt.
Introduction
Juhyeon Kim, Wojciech Jarosz and Adithya Pediredla present ToF ReSTIR, a reuse framework for time-of-flight rendering. Current time-of-flight methods cost heavily and scale poorly to complex moving scenes. Adapting ReSTIR reuse ideas to time-resolved light transport, they keep optical path lengths valid through path-length-aware shift mapping. That mapping is a geometric transform built on Newton's method, which adjusts reused paths to meet time-gating limits. The framework supports time-gated and transient rendering at interactive rates.
Catalog record
| CREATOR | Juhyeon Kim, Wojciech Jarosz, Adithya Pediredla |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 81650 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.11536
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
ToF ReSTIR: Time-of-Flight Rendering with Spatio-temporal Reservoir Resampling 251 Journal: TOG Journal: TOG Volume: 45 4 89 7 DOI: 10.1145/3811299 CCS: Computing methodologies Ray tracing CCS: Computing methodologies Computational photography Juhyeon Kim, Wojciech Jarosz, Adithya Pediredla Abstract . We present a novel spatio-temporal reuse framework for time-resolved light transport, enabling efficient Monte Carlo rendering of time-of-flight (ToF) phenomena such as time-gated imaging and transient light capture. Existing ToF rendering methods are computationally expensive, scale poorly to complex dynamic scenes, and are therefore unsuitable for applications with strict latency constraints. To address this limitation, we draw inspiration from ReSTIR , a reuse-based technique for steady-state real-time rendering, and adapt its core principles to interactive-rate ToF simulation. However, naively applying existing ReSTIR methods to ToF rendering leads to severe inefficiency, as reused paths frequently violate optical path-length constraints and thus contribute little or no signal. We overcome this challenge by introducing a path reuse formulation that explicitly enforces physically valid optical path lengths. The key idea is path-length-aware shift mapping , a geometric transformation based on Newton’s method that adjusts reused light paths to satisfy temporal gating constraints, inspired by specular manifold exploration in steady-state caustics rendering. The resulting framework substantially improves the efficiency of ToF rendering across a wide range of scenarios, including complex scenes with glossy or specular materials and dynamic motion. Our method supports both time-gated and transient rendering at interactive frame rates, enabling simulation under practical latency constraints. We demonstrate the effectiveness of our approach through two downstream applications, including shape reconstruction and navigation. Figure 1. We propose a real-time Time-of-Flight (ToF) rendering method inspired by ReSTIR, a state-of-the-art technique for steady-state real-time rendering, augmented with a novel path-length-aware shift mapping. (A) Compared to conventional path tracing and naive path reuse adapted from steady-state ReSTIR ( 38 ) , our method produces improved visual results for both time-gated image rendering and transient histogram rendering in dynamic scenes at interactive frame rates. (B) We further demonstrate two interactive downstream applications enabled by our renderer. For non-line-of-sight (NLOS) reconstruction, we recover hidden voxel geometry using temporally focused imaging ( 46 ) , where time-gated image simulation with our method reliably reconstructs the shape of the teapot. We also simulate NLOS navigation tasks ( 63 ) , where the reduced noise in our rendered transient histograms enables robust neural network inference and reliable path planning, whereas baseline methods fail to produce safe paths due to noise. † † cc-license: by 1. Introduction Light travels fast, but not infinitely fast—and even small delays in a photon’s arrival carry rich information about the physical world. Time-of-flight (ToF) imaging leverages this principle by emitting controlled illumination and measuring when photons return to the sensor after interacting with the scene. These time-resolved measurements capture both direct reflections from visible surfaces and multiply scattered transport, often originating from partially or fully hidden objects. As a result, ToF imaging enables advanced sensing capabilities—such as non-line-of-sight reconstruction and material characterization in challenging environments—that are inaccessible to conventional steady-state intensity cameras. Time-of-flight rendering provides a physically grounded way for simulating measurements from time-resolved imaging systems and plays an important role in understanding, validating, and developing modern ToF sensors. Such simulations support offline analysis and algorithm development, and enable the generation of ground-truth datasets that may be difficult or impractical to acquire experimentally. Beyond these offline uses, improved rendering performance enables ToF-based systems to be evaluated in interactive and decision-driven settings. For example, applications such as real-time reconstruction, navigation, and closed-loop perception require simulation to operate within strict latency budgets in order to be applicable. Efficient and physically accurate ToF rendering therefore, can extend the scope of time-resolved simulation from offline analysis to interactive sensing and perception tasks. Despite recent progress, existing ToF rendering methods face a fundamental trade-off. Physically accurate approaches ( 23 ; 42 ; 47 ; 41 ) are often computationally expensive and limited to offline use, while faster methods ( 20 ; 34 ) rely on simplified assumptions such as reduced geometry, materials, or light transport. As a result, current techniques struggle to achieve both efficiency and physical generality in dynamic, large-scale scenes. Motivated by these limitations, it is natural to ask whether techniques developed for real-time physically based steady-state rendering (e.g., games or commercial rendering) can be adapted to support interactive and scalable ToF simulation. In particular, reuse-based methods from Monte Carlo path tracing provide a promising starting point, as they aim to reduce computational cost while preserving physical correctness. Reservoir-based spatio-temporal resampling, so-called ReSTIR ( 4 ) and related approaches ( 44 ; 38 ) are representative methods in this direction, demonstrating how aggressive spatio-temporal reuse can scale path tracing to dynamic and interactive settings. However, these methods are fundamentally designed for steady-state rendering and do not account for optical path length. When applied directly to ToF rendering, reused paths frequently violate temporal constraints, leading to invalid contributions ( Fig. 2 ). To address these challenges, we introduce a path-length-aware spatio-temporal reuse framework for ToF rendering. The key idea is path-length-aware shift mapping , which geometrically adjusts a reused path so that its optical path length matches the active time gate (red arrow in Fig. 2 ). We formulate this adjustment as a constrained root-finding problem and solve it using Newton’s method. Our approach is inspired by manifold exploration techniques for specular transport ( 22 ; 36 ; 18 ) , but is adapted to enforce path-length constraints specific to ToF rendering. By incorporating path-length awareness into ReSTIR-style reuse, our method enables scalable and efficient simulation of ToF sensors in complex, dynamic environments. Our reuse strategy supports both per-frame image and transient histogram rendering, and we demonstrate its effectiveness in two downstream NLOS applications: shape reconstruction and inference-based navigation ( Fig. 1 ). The code and data are available on the project page 1 1 1 https://juhyeonkim95.github.io/project-pages/tof_restir . Figure 2. Overview of proposed path-length-aware reuse strategies. (A) Naive path reuse from conventional steady-state ReSTIR (e.g., hybrid shift-mapping by 38 ) ends up going outside of the valid time gate. To address this issue, we introduce path-length-aware shift mapping, illustrated by the red arrow. (B) Illustration of a base path in the left pixel (blue), the naively reused path in the right pixel (orange), and the corrected path produced by our method (green), which satisfies the temporal constraint. 2. Related Works 2.1. Time-of-Flight Imaging Here, we review representative ToF imaging systems that acquire either transient measurements—sequences of time-resolved images—or time-gated measurements, which capture a single image within a narrow temporal window. 24 provide a more comprehensive review. One common approach employs high-speed optical gating to selectively detect photons that arrive within short temporal windows after pulsed illumination. Intensified CCD/CMOS cameras achieve this using a fast-gating image intensifier, allowing selective capture of early or late light within sub-nanosecond windows ( 7 ) . By shifting these gates over repeated acquisitions, a full transient profile can be reconstructed. Streak cameras achieve even finer temporal resolution by mapping photon arrival times to spatial deflections, enabling picosecond-scale measurements ( 57 ; 58 ) . Single-photon avalanche diode (SPAD) sensors have emerged as compact, scalable detectors operating at the level of individual photons ( 64 ) . In time-correlated single-photon