Advances in 4D Representation: Geometry, Motion, and Interaction
Mingrui Zhao 1 , Sauradip Nag 1 , Kai Wang 1 , Aditya Vora 1 , Guangda Ji 1 , Peter Chun 2 , Ali Mahdavi-Amiri 1 , Hao Zhang 1 1 Simon Fraser University, 2 University of Alberta · 2025
Read Advances in 4D Representation: Geometry, Motion, and Interaction by Mingrui Zhao 1 , Sauradip Nag 1 , Kai Wang 1 , Aditya Vora 1 , Guangda Ji 1 , Peter Chun 2 , Ali Mahdavi-Amiri 1 , Hao Zhang 1 1 Simon Fraser University, 2 University of Alberta (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-19255.txt.
Introduction
Mingrui Zhao and seven co-authors survey four-dimensional content. Such content is three-dimensional geometry that moves over time. It can also interact. Neural fields and motion learning have sped the field up fast. Three-dimensional generative tools helped too. The right representation decides what a system can capture.
The survey does not list every paper. It picks representative works instead. Each choice shows a strength and a challenge. Data and application conditions shape both. Readers get a map of trade-offs. The map serves builders of dynamic scenes in graphics and vision.
Catalog record
| CREATOR | Mingrui Zhao 1 , Sauradip Nag 1 , Kai Wang 1 , Aditya Vora 1 , Guangda Ji 1 , Peter Chun 2 , Ali Mahdavi-Amiri 1 , Hao Zhang 1 1 Simon Fraser University, 2 University of Alberta |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 112 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2510-19255
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Advances in 4D Representation: Geometry, Motion, and Interaction Mingrui Zhao 1 , Sauradip Nag 1 , Kai Wang 1 , Aditya Vora 1 , Guangda Ji 1 , Peter Chun 2 , Ali Mahdavi-Amiri 1 , Hao Zhang 1 1 Simon Fraser University, 2 University of Alberta Abstract We present a survey on 4D generation and reconstruction, a fast-evolving subfield of computer graphics whose developments have been propelled by recent advances in neural fields, geometric and motion deep learning, as well 3D generative artificial intelligence (GenAI). While our survey is not the first of its kind, we build our coverage of the domain from a unique and distinctive perspective of 4D representations , to model 3D geometry evolving over time while exhibiting motion and interaction. Specifically, instead of offering an exhaustive enumeration of many works, we take a more selective approach by focusing on representative works to highlight both the desirable properties and ensuing challenges of each representation under different computation, application, and data scenarios. The main take-away message we aim to convey to the readers is on how to select and then customize the appropriate 4D representations for their tasks. Organizationally, we separate the 4D representations based on three key pillars: geometry , motion , and interaction . Our discourse will not only encompass the most popular representations of today, such as neural radiance fields (NeRFs) and 3D Gaussian Splatting (3DGS), but also bring attention to relatively under-explored representations in the 4D context, such as structured models and long-range motions. Throughout our survey, we will reprise the role of large language models (LLMs) and video foundational models (VFMs) in a variety of 4D applications, while steering our discussion towards their current limitations and how they can be addressed. We also provide a dedicated coverage on what 4D datasets are currently available, as well as what is lacking, in driving the subfield forward. cleveref † † teaser: Representative applications for three key pillars of 4D Representation : (a) (Geometry) mesh animation [ CZTW25 ] , (b) (Geometry) dynamic point map reconstruction [ WZZ∗25 ] , (c) (Motion) 4D inbetweening [ NCOZMA25 ] ,(d) (Motion) 4D semantic segmentation [ MRH22 ] , (e) Human-scene interaction [ LYLW24 ] ,(f) Human-object interaction [ LWL23 ] . 1 Introduction Reconstruction and generation of 4D data, i.e., 3D geometry evolving over time while exhibiting motion and interaction, is becoming increasingly critical as computer graphics applications expand into domains requiring dynamic scene understanding, temporal modeling, and motion synthesis. From cinematic visual effects and immersive virtual reality to autonomous robotics, medical imaging, eCommerce and advertising, the ability to capture, represent, and manipulate 4D content has emerged as a fundamental challenge that bridges graphics, vision, and machine learning. The fourth dimension introduces complexities that extend far beyond simply concatenating spatial coordinates with temporal indices. Temporal coherence, motion continuity, topological changes, interaction dynamics, and the preservation of geometric, appearance, and physical properties across time present unique representational challenges that require careful consideration of both spatial and temporal encoding strategies. As the field matures, researchers have developed increasingly sophisticated approaches to handle these challenges, leading to a rich landscape of 4D representation schemes, each with distinct advantages and limitations. Recent surveys in this domain have primarily categorized methods by applications, approaches, and/or the granularity of the extracted scene signals. Cao et al. [ CLH∗25 ] provides a structured review of recent progress in reconstructing 4D spatial intelligence, which is defined as understanding 3D scenes and their evolution over time, advancing from extracting low-level geometric cues such as depth, pose, and point maps toward complex reasoning encompassing interactions and physics. This survey does highlight the growing importance of advanced 3D representations while focusing mainly on neural radiance fields (NeRFs) and 3D Gaussian splatting (3DGS). Along the same lines, Fan et al. [ FZZ∗25 ] presents a comprehensive survey of recent progress in dynamic scene representation and reconstruction, focusing on the transition from NeRF to 3DGS and their trade-offs. This second survey systematically categorizes the works covered based on motion representation paradigms including rigid, articulated, non-rigid, and hybrid motions, and examines strategies for modeling temporal changes in geometry and appearance. In addition, Miao et al. [ MLQ∗25 ] surveys the emerging field of 4D generation by introducing a taxonomy of low-level geometry representations (meshes, NeRFs, point clouds, and 3DGS), foundational techniques (diffusion models and score distillation sampling), and conditioning methods (text, images, videos, 3D inputs, and multimodal control). This survey further categorizes the algorithmic approaches (end-to-end, via generated data, implicit distillation, vs. explicit supervision) and highlights the growing range of applications for 4D generation. While the above surveys have advanced our understanding of the field’s breadth, they have not sufficiently covered all the relevant representations of geometry (especially higher-level structured representations), motion, and interaction, or addressed the fundamental question of why specific representations are chosen for particular 4D tasks and the motion/interaction mechanisms, nor have they provided comprehensive analysis of the inherent trade-offs that different representational choices impose on data preparation, method design, computational requirement, and achievable results. To close these gaps, our survey adopts a representation-centric perspective. Instead of offering an exhaustive enumeration of many works, we take a more selective approach by focusing on representative works to highlight both the desirable properties and ensuing challenges of each 4D representation under different computation, application, and data scenarios. The main take-away message we aim to convey to the readers is on how to select and then customize the appropriate 4D representations for their tasks. Organizationally, we separate the 4D representations based on three key pillars: geometry , motion , and interaction . We distinguish between structured representations, i.e., those that maintain explicit primitive or part delineations and relations, and unstructured representations, i.e., those that encode 4D content through implicit functions, point distributions, or learned feature spaces without explicit structural constraints. While the latter category encompasses the most popular representations of today, namely NeRFs and 3DGS, they are both built on rendering primitives, mainly for novel view synthesis. They are not best suited to prevalent 4D tasks such as modeling, editing, or interactions, for which more compact and structural representations, built on higher-level primitives, are more appropriate. Hence, we also discuss these representational choices even though they have been been relatively under-explored. Throughout our survey, we will reprise the role of large language models (LLMs) and video foundational models (VFMs) in a variety of 4D applications, while steering our discussion towards their current limitations and how they can be addressed. We also provide a dedicated coverage on what 4D datasets are currently available, as well as what is lacking, in driving the subfield forward. Our representation-centric analysis goes beyond traditional surveys in several key ways. First, we examine not only the capabilities of each representation but also their fundamental limitations and the specific 4D challenges they are designed to address. Second, we analyze how representational choices constrain and enable different architectural approaches for learning, predicting, and generating temporal dynamics. Third, we provide comprehensive coverage of motion modeling approaches, temporal consistency mechanisms, and the interplay between representation and motion characteristics aspects that have received limited attention in previous surveys despite their central importance to 4D applications. Furthermore, this survey addresses the growing importance