Real-time 3D Visualization of Radiance Fields on Light Field Displays 1769
Jonghyun Kim, Cheng Sun, Michael Stengel, Matthew Chan, Andrew Russell, Jaehyun Jung, Wil Braithwaite, Shalini De Mello, David Luebke · 2025
Read Real-time 3D Visualization of Radiance Fields on Light Field Displays 1769 by Jonghyun Kim, Cheng Sun, Michael Stengel, Matthew Chan, Andrew Russell, Jaehyun Jung, Wil Braithwaite, Shalini De Mello, David Luebke (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-18540.txt.
Introduction
Jonghyun Kim and eight co-authors speed up 3D scene rendering for light field screens in a 2025 paper. Such screens need dozens of views from slightly shifted spots at once, while radiance field scenes are heavy to draw even once. Naive per view drawing cannot run live, so the team reshaped the work around shared parts.
Their frame turns a scene into sweeping planes cached once, then blends them into all views in a single pass. The trick trades fine view based color for speed and a bit more memory. It works across scene forms with no retraining. On a Looking Glass screen the demo ran past two hundred frames per second across forty five views. Bench tests showed speed gains up to twenty two fold.
Catalog record
| CREATOR | Jonghyun Kim, Cheng Sun, Michael Stengel, Matthew Chan, Andrew Russell, Jaehyun Jung, Wil Braithwaite, Shalini De Mello, David Luebke |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 40 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2508-18540
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Real-time 3D Visualization of Radiance Fields on Light Field Displays 1769 Jonghyun Kim, Cheng Sun, Michael Stengel, Matthew Chan, Andrew Russell, Jaehyun Jung, Wil Braithwaite, Shalini De Mello, David Luebke Abstract . Radiance fields have revolutionized photo-realistic 3D scene visualization by enabling high-fidelity reconstruction of complex environments, making them an ideal match for light field displays. However, integrating these technologies presents significant computational challenges, as light field displays require multiple high-resolution renderings from slightly shifted viewpoints, while radiance fields rely on computationally intensive volume rendering. In this paper, we propose a unified and efficient framework for real-time radiance field rendering on light field displays. Our method supports a wide range of radiance field representations—including NeRFs, 3D Gaussian Splatting, and Sparse Voxels—within a shared architecture based on a single-pass plane sweeping strategy and caching of shared, non-directional components. The framework generalizes across different scene formats without retraining, and avoids redundant computation across views. We further demonstrate a real-time interactive application on a Looking Glass display, achieving 200+ FPS at 512p across 45 views, enabling seamless, immersive 3D interaction. On standard benchmarks, our method achieves up to 22 × \times speedup compared to independently rendering each view, while preserving image quality. Figure 1 . We present a unified framework for real-time radiance field rendering on light field displays, supporting NeRFs (N2LF), 3D Gaussians (G2LF), and sparse voxels (V2LF) via a single-pass plane sweep. (Left) Render time comparison across view counts on the MipNeRF-360 dataset ( Barron et al. 2022 ) , measured at 512p on an RTX 5090. G2LF and V2LF maintain real-time performance (>60 FPS at N_{\mathrm{chunk}} =64) beyond 90 views. (Right) Captured results from a Looking Glass Go display, using a motorized rotational stage, confirm high-quality 3D reconstruction up to 22× speedup over per-view rendering. 1. Introduction Recent advancements in radiance fields have significantly improved both the quantity and quality of 3D content. Radiance fields represent 3D scenes by encoding density and color values across spatial coordinates and view directions, enabling photorealistic reconstructions of complex scenes. Neural Radiance Fields (NeRF) have enabled continuous view synthesis from sparse input images, making it possible to reconstruct complex 3D scenes with high precision ( Mildenhall et al. 2021 ; Müller et al. 2022 ) . Radiance fields now encompass a variety of representations for improving rendering efficiency for real-time applications, including rasterization-based methods ( Kerbl et al. 2023 ; Sun et al. 2024 ) and explicit scene structures ( Takikawa et al. 2021 ; Sun et al. 2022 ) that avoid dense sampling. Like other 3D content, radiance fields are most effectively visualized using 3D displays. Their ability to represent complex 3D structures aligns naturally with the capabilities of light field displays, which optically reconstruct the light rays of 3D scenes. Recent commercially available light field displays offer high spatial and angular resolution, enabling immersive 3D visualization ( ( Leia Inc. 2025 ) , ( Sony Electronics Inc. 2025 ) , ( Looking Glass Factory Inc. 2025 ) ). These displays provide binocular disparity and motion parallax, leveraging human depth perception to allow users to naturally perceive 3D structures. This integration, however, introduces significant computational challenges. First, light field displays require high-resolution rendering from many (45+) slightly shifted viewpoints. Second, generating views from radiance fields, even with fast rasterization-based methods like Gaussian Splatting, remains more computationally expensive than traditional graphics pipelines. Light field displays fundamentally face substantial computational overhead in rendering due to their unique optical design. Unlike conventional single-view 2D displays, they require the generation of multiple perspective views to reconstruct the full light field, necessitating a dense array of rays projected at precise angles. This significantly increases the computational burden compared to single-view 2D displays. Furthermore, precise optical alignment between the display panel and the lens array is critical; even minor angular or spatial misalignment during manufacturing can lead to incorrect ray-to-subpixel mappings, degrading visual quality (see section 3 for details). These misalignments demand a per-device calibration process to ensure accurate ray alignment, further complicating the rendering pipeline. Radiance fields face inherent computational challenges, particularly due to their reliance on volumetric rendering, where density and color values must be evaluated along each ray. To reduce these costs, recent approaches employ explicit/hybrid data structures and rasterization-based pipelines to accelerate rendering, such as 3D Gaussian Splatting (3DGS) ( Kerbl et al. 2023 ) and sparse voxels ( Sun et al. 2024 ) . However, when targeting light field displays, these methods still require rendering many slightly shifted viewpoints, making repeated sampling inefficient and redundant. Addressing this redundancy is essential to enabling real-time radiance field rendering for light field applications. In this paper, we present a unified framework for real-time radiance field rendering on light field displays. Our framework supports a wide range of radiance field representations—including NeRFs, 3DGS, and sparse voxels—within a unified rendering architecture. The core of our method is a single-pass plane sweeping strategy that enables efficient view synthesis while preserving image quality. We implement this framework in three variants: NeRF-to-light-field (N2LF), 3DGS-to-light-field (G2LF), and sparse-voxels-to-light-field (V2LF), corresponding to different input representations. We demonstrate the effectiveness of our approach through an interactive application running on a commercial light field display, enabling smooth and immersive real-time 3D visualization of radiance field. Our GL-based interactive demo achieves 228 FPS (22 \times faster compared to native quilt rendering, see section 5.3 ) for 45-view, 512 \times 910 light field images on the bicycle scene using a single NVIDIA RTX 5090 graphics card. (see supplementary video) The contributions of this paper are as follows: • We propose a unified and efficient multi-view rendering framework for radiance fields for light field displays, supporting both implicit (NeRF) and explicit/hybrid (3DGS, sparse voxel) representations without retraining. • Our method achieves real-time performance (>60 FPS for 90+ views, up to 228 FPS for 45 views) by minimizing redundant sampling through a single-pass plane sweeping strategy. • We develop an OpenGL-based interactive 3DGS renderer for a commercial light field display, enabling real-time 3D visualization and achieving a 3.6 \times speedup over our Python/CUDA baseline. 2. Related Work Multi-view rendering Techniques such as multi-view point splatting ( Hübner et al. 2006 ) , single-pass multi-view rendering ( Hübner et al. 2007 ) , and unstructured lumigraph rendering ( Buehler et al. 2001 ) , as well as hardware-level instancing methods ( NVIDIA Corporation 2025 ; Unterguggenberger et al. 2020 ) , reduce rendering cost across nearby views. However, these methods are tailored to traditional 3D content, such as meshes or point clouds, and are difficult to apply to radiance fields, where sharing computation across nearby views remains challenging. Our method addresses this by providing a unified solution that supports both implicit and explicit representations, enabling single-pass rendering of high-quality multi-view outputs. Depth-guided view synthesis Classical methods such as depth-image-based rendering ( Fehn 2004 ) , layered depth images ( Shade et al. 1998 ) , and multiplane images ( Zhou et al. 2018 ) synthesize novel views by projecting RGB-D inputs onto proxy geometry or discrete depth layers. These efficient methods rely on given depth and fixed textures, often leading to disocclusion artifacts and limited geometric fidelity ( Sun et al. 2010 ) . While our approach shares structural similarities, we