ArmGS: Composite Gaussian Appearance Refinement for Modeling Dynamic Urban Environments
Guile Wu, Dongfeng Bai, Bingbing Liu · 2025
Read ArmGS: Composite Gaussian Appearance Refinement for Modeling Dynamic Urban Environments by Guile Wu, Dongfeng Bai, Bingbing Liu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-03886.txt.
Introduction
Guile Wu, Dongfeng Bai, and Bingbing Liu model busy city streets for self-driving tests, in this 2025 paper. Car makers need photo-real virtual streets to train and check driving software safely. A recent technique builds scenes from many soft 3D blobs, called Gaussian splatting, and renders them fast. Yet moving traffic and shifting viewpoints expose flaws: cars and lights can look off from new angles or in the next frame.
Their method, ArmGS, layers several blob sets and refines how each looks at both fine and coarse detail. That composite setup tracks changes between frames and across camera spots. The goal is a street model that stays sharp while cars move and viewpoints swing. More faithful simulators mean driving stacks can be tested harder before they meet real roads.
Catalog record
| CREATOR | Guile Wu, Dongfeng Bai, Bingbing Liu |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 36 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2507-03886
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
ArmGS: Composite Gaussian Appearance Refinement for Modeling Dynamic Urban Environments Guile Wu, Dongfeng Bai, Bingbing Liu Abstract This work focuses on modeling dynamic urban environments for autonomous driving simulation. Contemporary data-driven methods using neural radiance fields have achieved photorealistic driving scene modeling, but they suffer from low rendering efficacy. Recently, some approaches have explored 3D Gaussian splatting for modeling dynamic urban scenes, enabling high-fidelity reconstruction and real-time rendering. However, these approaches often neglect to model fine-grained variations between frames and camera viewpoints, leading to suboptimal results. In this work, we propose a new approach named ArmGS that exploits composite driving Gaussian splatting with multi-granularity appearance refinement for autonomous driving scene modeling. The core idea of our approach is devising a multi-level appearance modeling scheme to optimize a set of transformation parameters for composite Gaussian refinement from multiple granularities, ranging from local Gaussian level to global image level and dynamic actor level. This not only models global scene appearance variations between frames and camera viewpoints, but also models local fine-grained changes of background and objects. Extensive experiments on multiple challenging autonomous driving datasets, namely, Waymo, KITTI, NOTR and VKITTI2, demonstrate the superiority of our approach over the state-of-the-art methods. Figure 1 : An illustration of our ArmGS for modeling urban scenes in autonomous driving. Our approach is capable of modeling fine-grained changes of background scenes and objects across frames and camera viewpoints. 1 Introduction Autonomous driving has made remarkable advancements in recent years, but how to effectively validate the safety and reliability of an autonomous driving system remains an open question. In real-world driving scenes, there are many corner cases, such as collisions, which cannot be easily collected, but are critical to the safety of an autonomous driving system. Simulation is one of the most useful ways to alleviate this problem. It enables the reconstruction and generation of various challenging driving scenes for downstream closed-loop evaluation. Traditionally, virtual-engine-based methods, such as CARLA Dosovitskiy et al. 2017 , use classic graphics rendering systems for driving scene simulation, but the simulated scenes often have significant gaps from the real-world ones. On the other hand, data-driven simulation enables photorealistic driving scene modeling in a low-cost and efficient way Yan et al. 2024 ; Yang et al. 2024 ; Yang et al. 2023 ; Wu et al. 2023 ; Huang et al. 2024 . With the great success of Neural Radiance Fields (NeRFs) for neural rendering, some researchers have adapted NeRFs for scene modeling in autonomous driving Yang et al. 2024 ; Yang et al. 2023 ; Wu et al. 2023 ; Ost et al. 2021 . Although NeRF-based methods have shown great potential for reconstructing realistic driving scenes, they mostly suffer from long training time and fail to achieve real-time rendering. More recently, some researchers have resorted to 3D Gaussian Splatting (3DGS) Kerbl et al. 2023 to learn explicit 3D scene representations for driving scene modeling Yan et al. 2024 ; Huang et al. 2024 ; Ren et al. 2024b ; Chen et al. 2023 ; Zhou et al. 2024b . Due to the efficiency of explicit representations and differentiable tile rasterizers, 3DGS-based methods have shown promising results for high-fidelity reconstruction and real-time rendering. However, existing approaches for driving scene modeling mostly neglect to model fine-grained driving scene variations between frames and camera viewpoints 1 1 1 Throughout this work, camera viewpoints refer to different locations of the same camera or different cameras . In autonomous driving simulation, scene images are captured in the wild along the movement of the ego vehicle. This unavoidably brings significant appearance changes of scenes and objects across frames and camera viewpoints due to lighting and camera exposure. Thus, failing to model these variations can lead to missing fine-grained details and suboptimal simulation results. In this work, we propose to exploit composite 3D G aussian S platting with A ppearance R efine M ent (ArmGS) from multiple granularities for dynamic urban environments modeling in autonomous driving. Our approach constructs a composite driving scene model based on 3DGS representations and employs a multi-level appearance modeling scheme to optimize a set of transformation parameters for composite Gaussian refinement. Specifically, we represent a driving scene with composite 3DGS and refine them at local Gaussian level, global image level and dynamic actor level, respectively. For local Gaussian level refinement, we extract latent Gaussian appearance representation to learn Gaussian-wise transformation parameters for Gaussian appearance refinement. This helps to learn local fine-grained appearance variations in the 3D Gaussian space. For global image level refinement, we extract latent image appearance representation to learn global image-wise transformation parameters for image color refinement. This helps to learn global image-wise appearance changes across frames and camera viewpoints. Moreover, to model dynamic variations of moving road actors (such as vehicles), we employ a light-weight spatial-temporal deformation model to transform dynamic actor Gaussians across time. This multi-level appearance modeling scheme allows for refining the composite driving scene Gaussians at multiple granularities. Consequently, scene and road actors appearance variations are collectively modeled to reconstruct richer fine-grained cues of driving scenes. Our experiments on multiple challenging autonomous driving datasets, namely, Waymo Sun et al. 2020 , KITTI Geiger et al. 2012 , NOTR Yang et al. 2024 and VKITTI2 Cabon et al. 2020 , for driving scene modeling and novel view synthesis show that our approach not only reconstructs global scene appearance variations between frames and camera viewpoints, but also models local fine-grained changes of background and dynamic objects. An illustration is shown in Fig. 1 . In summary, our contributions are: • Our method is the first method that proposes to explicitly embed multi-granularity appearances to model fine-grained changes for dynamic urban environments modeling. This fills a gap left by both existing NeRF and Gaussian splatting methods. • Our method uses a set of transformation parameters for composite Gaussian refinement, which is simple and effective and will not break the differentiability of splatting. • Our experiments on Waymo, KITTI, NOTR and VKITTI2 show the superiority of our method over the state-of-the-art methods. Figure 2 : An overview of our ArmGS approach. Our approach refines composite driving scene Gaussians with appearance modeling at multiple granularities, ranging from local Gaussians level to global images level and dynamic actors level. The modules for local level refinement, global level refinement and actor level refinement are indicated in yellow, green and blue, respectively. 2 Related Work 3D Gaussian Splatting. 3D Gaussian Splatting (3DGS) Kerbl et al. 2023 is a recently proposed real-time rendering approach. Unlike NeRFs that employ a continuous radiance field to implicitly represent 3D scenes, 3DGS represents a 3D scene with a collection of explicit 3D Gaussian primitives and utilizes splatting-based rasterization Zwicker et al. 2001 to project 3D Gaussians to 2D image space. Since the success of Kerbl et al. 2023 , researchers have made many efforts to improve 3DGS from different aspects Yu et al. 2024 ; Fan et al. 2024 ; Zhang et al. 2024 ; Dahmani et al. 2024 ; Lyu et al. 2024 , such as anti-aliasing Yu et al. 2024 , compression Fan et al. 2024 , surface reconstruction Lyu et al. 2024 , etc. Among them, Dahmani et al . Dahmani et al. 2024 propose to model image