ViVo: A Dataset for Volumetric Video Reconstruction and Compression

Adrian Azzarelli, Ge Gao, Ho Man Kwan · 2025

Read ViVo: A Dataset for Volumetric Video Reconstruction and Compression by Adrian Azzarelli, Ge Gao, Ho Man Kwan (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-00558.txt.

Introduction

Adrian Azzarelli and co-authors release ViVo, a data set for volumetric video, in a 2025 paper. Volumetric video captures a scene in three dimensions, so viewers can move around inside it. Rebuilding and squeezing such video needs varied, realistic test data. Older data sets lack the range of content found in real production work.

ViVo broadens what diversity means for this field. Scenes vary in human traits like skin and hair, and in tricky sights like glass, shine, and liquid. Each sequence packs many paired color and depth views, plus calibration, sound, masks, and point clouds. The team benchmarked leading rebuilding and compression methods on it. Results showed the set is hard for current tools.

Catalog record

CREATORAdrian Azzarelli, Ge Gao, Ho Man Kwan
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE46 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2506-00558

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

ViVo: A Dataset for Volumetric Video Reconstruction and Compression Thanks: Manuscript submitted in May 2025. Thanks: Adrian Azzarelli, Ge Gao, Ho Man Kwan, Fan Zhang, Nantheera Anantrasirichai and David Bull are with the Visual Information Lab, University of Bristol, Bristol, BS1 5DD, U.K. (e-mail: {a.azzarelli, ge1.gao, hm.kwan, fan.zhang, n.anantrasirichai, dave.bull}@bristol.ac.uk). Ollie Moolan-Feroze is with Condense Reality Ltd, 1 Canon’s Road, Bristol, BS1 5TX, U.K. (e-mail: [email protected]) Thanks: The authors acknowledge the funding from Innovate UK (10107058) and the UKRI MyWorld Strength in Places Programme (SIPF00006/1). Adrian Azzarelli, Ge Gao, Ho Man Kwan Abstract As research on neural volumetric video reconstruction and compression flourishes, there is a need for diverse and realistic datasets, which can be used to develop and validate reconstruction and compression models. However, existing volumetric video datasets lack diverse content in terms of both semantic and low-level features that are commonly present in real-world production pipelines. In this context, we propose a new dataset, ViVo, for VolumetrIc VideO reconstruction and compression. The dataset is faithful to real-world volumetric video production and is the first dataset to extend the definition of diversity to include both human-centric characteristics (skin, hair, etc.) and dynamic visual phenomena (transparent, reflective, liquid, etc.). Each video sequence in this database contains raw data including fourteen multi-view RGB and depth video pairs, synchronized at 30FPS with per-frame calibration and audio data, and their associated 2-D foreground masks and 3-D point clouds. To demonstrate the use of this database, we have benchmarked three state-of-the-art (SotA) 3-D reconstruction methods and two volumetric video compression algorithms. The obtained results evidence the challenging nature of the proposed dataset and the limitations of existing datasets for both volumetric video reconstruction and compression tasks, highlighting the need to develop more effective algorithms for these applications. The database and the associated results are available at https://vivo-bvicr.github.io/ I Introduction Aacquiring diverse and representative volumetric video data is an active challenge that is key to computer vision research. Over the years, datasets such as ZJU MoCap and CMU Panoptic [ 1 , 2 ] have facilitated advances in numerous fields, including dynamic 3-D reconstruction and multi-view video (MVV) compression. Consequently, as research related to creative volumetric video reconstruction and compression emerges, the demand for effective datasets also grows. The volumetric video production pipeline begins with MVV capture, where videos and other sensor data are collected from various viewing angles and positions around a target scene (e.g. a performer). Based on the captured data, a realistic dynamic 3-D scene is then synthesized, compressed, and transmitted to an audience for consumption. However, most existing MVV datasets focus on semantic scenarios rather than on different low-level materials and texture types (e.g. dynamic textures and transparent materials), which are important for assessing performance on the target tasks (reconstruction or compression). Many datasets also do not simulate the real volumetric video production scenarios. As they focus only on human-centric volumetric reconstruction, the capture area is designed to physically isolate the actor from the film crew to minimize background noise. In real production scenarios, physically isolating the actor is not practical as it impacts the quality of the performance. Hence, removing/minimizing dynamic background noise (e.g., the film crew moving around) is a major challenge that is not well represented by existing datasets. (a) RGB Data + Zoomed Image (b) Depth Data (c) Mask + Point Cloud Data (d) Camera Calibration + Train-Test Split (e) Multi-view Sample Sequence Fig. 1 : We showcase the scenes and relevant data associated with the NVS and MVV compression experiments. (a-b) is the source data. (c) is the supplementary data. (d) shows the camera poses and indicates in green the position of the test cameras. (e) illustrates a sample sequence from two cameras To further drive research linked to V olumetr i c V ide o reconstruction and compression, this paper contributes ViVo , a diverse and professionally collected dataset of MVV sequences that are faithful to the volumetric media production process and capture a high variety of dynamic objects, people, textures and practical effects. This database contains 32 unique volumetric video sequences, lasting between 60 and 120 seconds. Per sequence, each frame is associated with 14 RGB views at a spatial resolution of 2560 \times 1440 pixels, and synchronized 14 depth views at 650 \times 576 pixels, as well as their corresponding camera poses, intrinsic data and audio signals (as shown in Fig. 1 ). These metadata are important in real production applications but are not supported in existing datasets. The collected content reflects challenges that are present in both the real and research settings, in particular including various dynamic scenes such as fire, liquid, inflatable costumes, and balloons. As the dataset focuses on human activities, ViVo also captures actors with different ages, genders, hair styles, skin colors, and clothes. Based on the collected data, we also provide synthesized per-frame colored point clouds (up to 155 million points per second), dynamic 2-D foreground masks, and 3-D box masks for various processing tasks. To ensure high-quality 2-D masks are generated, we integrate SAM2 [ 3 ] into a GUI that allows users to re-synthesize our masks or create their own, as well as apply erosion and dilation. Manually drawn static masks are also provided and segment the staging area from the dynamic background. To demonstrate the use of this dataset for volumetric video reconstruction and compression, we benchmarked three state-of-the-art (SotA) models on novel view synthesis (NVS) and three SotA methods for MVV compression. Contrary to findings in studies that evaluate dense-view NVS, the sparse-view NVS experiments show that newer approaches fail to capture fine details both spatially and temporally. The results also indicate that classical image quality metrics are not effective for evaluating visual performance as they are prone to temporal bias. The experimental findings in MVV compression reveal that the evaluated baseline methods struggle to reconstruct fine-grained or highly dynamic textures accurately - an issue that becomes particularly pronounced under low-bitrate conditions. The main contributions of this work are summarized as follows. • As far as we are aware, ViVo is the first multi-view human video dataset which simulates real production settings, and covers a diverse range of people and dynamic visual phenomena. It is also the only dataset to provide per-frame calibration, RGB and depth data for each captured scenario. • Associated with this dataset, we further provide a set of open-source data processing tools that allow users to: (i) utilize a SotA video segmentation method to produce high quality per-frame 2-D dynamic masks, and (ii) generate colored point clouds with up to 155 million points per frame. • We conduct experiments that benchmark both SotA dynamic NVS models and SotA MVV compression models on the new database. A detailed analysis is provided on the benchmark results which identifies potential future research directions. The remainder of this paper is organized as follows. Section II briefly summarizes the related work in volumetric video reconstruction and compression, in particular focusing on existing databases for these tasks. Section III provides a detailed description of the database, including captured scenarios, acquisition configurations, and the associated data and tools. Based on the dataset, the designed benchmark experiments are described in Section IV for both NVS and MVV tasks,

Read the complete text