YCB-Ev SD: Synthetic event-vision dataset for 6DoF object pose estimation

Pavel Rojtberg, Julius Kühn · 2025

Read YCB-Ev SD: Synthetic event-vision dataset for 6DoF object pose estimation by Pavel Rojtberg, Julius Kühn (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-11344.txt.

Introduction

Pavel Rojtberg and Julius Kühn introduce YCB-Ev SD, a synthetic data set of event-camera recordings for estimating the full 3D pose of objects. Event cameras report pixel changes as they happen and suit fast motion, but the field lacks the large synthetic resources that frame-based vision enjoys. The set holds fifty thousand short event sequences rendered from physically based scenes of standard test objects.

Generation used simulated linear camera motion for full scene coverage, including background activity. A systematic study of event encodings for neural inference found that time surfaces with linear decay and two-channel polarity coding beat exponential decay and single-channel forms by wide margins. Polarity carried the largest share of the gains.

Catalog record

CREATORPavel Rojtberg, Julius Kühn
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE14 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2511-11344

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

YCB-Ev SD: Synthetic event-vision dataset for 6DoF object pose estimation Pavel Rojtberg, Julius Kühn Abstract We introduce YCB-Ev SD, a synthetic dataset of event-camera data at standard definition (SD) resolution for 6DoF object pose estimation. While synthetic data has become fundamental in frame-based computer vision, event-based vision lacks comparable comprehensive resources. Addressing this gap, we present 50,000 event sequences of 34 ms duration each, synthesized from Physically Based Rendering (PBR) scenes of YCB-Video objects following the Benchmark for 6D Object Pose (BOP) methodology. Our generation framework employs simulated linear camera motion to ensure complete scene coverage, including background activity. Through systematic evaluation of event representations for CNN-based inference, we demonstrate that time-surfaces with linear decay and dual-channel polarity encoding achieve superior pose estimation performance, outperforming exponential decay and single-channel alternatives by significant margins. Our analysis reveals that polarity information contributes most substantially to performance gains, while linear temporal encoding preserves critical motion information more effectively than exponential decay. The dataset is provided in a structured format with both raw event streams and precomputed optimal representations to facilitate immediate research use and reproducible benchmarking. The dataset is publicly available at https://huggingface.co/datasets/paroj/ycbev_sd . Figure 1 : Sythethic RGB renderings (left), matching event histogram (middle), and depth image (right) from the our dataset. Throughout this paper, the polarity of the events is represented as blue (falling) and green (rising). 1 Introduction and related work Real-time object detection and 6D pose estimation are critical capabilities for applications in augmented reality, virtual reality, and robotics. The field’s progress is benchmarked by the BOP Challenge [ 11 ] , which annually ranks algorithms and publishes a leaderboard. This challenge utilizes a variety of datasets that differ in captured modality (e.g., RGB, depth) and object types (e.g., household, industrial). Since 2020, the challenge has included 50,000 synthetic, physically based rendering (PBR) images, which have proven highly beneficial. These images provide precise annotations and a wide variety of lighting conditions and scene arrangements, often boosting performance beyond using real data alone. Among the datasets used in the BOP challenge, the YCB-Video (YCB-V) dataset [ 13 ] is particularly notable. It not only provides 3D models for generating synthetic renderings but also allows researchers to obtain the corresponding physical objects from the YCB organizers [ 1 ] . Event-based, or neuromorphic, cameras offer a novel sensing modality with advantages over traditional frame-based cameras, including high temporal resolution, high dynamic range, and low power consumption. Instead of capturing full frames at a fixed rate, they output a sparse, asynchronous stream of "events." Each event is triggered at a pixel when the intensity change exceeds a threshold, encoding the location, timestamp, and polarity (sign) of the change (see Figure 1 ). Dataset object classes frames high-res 1 real data synthetic data YCB-Ev [ 12 ] 21 13,851 ✓ ✓ ✗ E-POSE [ 5 ] 13 333,357 ✗ ✓ ✗ YCB-Ev SD (ours) 21 50,000 ✗ ✗ ✓ Table 1 : Overview of related YCB-based event datasets. Our dataset is distinguished as the only one providing synthetic event data. 1 High-resolution is defined as \geq HD (1280x720px) Table 1 provides an overview of existing YCB-based event datasets. Current datasets primarily focus on real event data, mirroring the initial approach taken with RGB-D data. For synthetic event data dedicated to object pose estimation, to the best of our knowledge, only one satellite-centric dataset exists [ 14 ] . However, this dataset uses simple OpenGL rendering instead of more realistic PBR, and its scenes contain no background and only one satellite per image. In this work, we align with the BOP challenge’s methodology for RGB data by providing 50,000 synthetic event sequences generated from PBR scenes. Existing real event datasets offer either a low resolution of 346x260px [ 5 ] or a high HD resolution of 1280x720 [ 12 ] . Recent studies [ 4 ] suggest that high-resolution event cameras can exhibit lower task performance in low-light conditions or during fast motion due to increased temporal noise. Specifically, [ 4 ] demonstrated that standard definition (SD) cameras (640x360) outperformed HD cameras in camera pose tracking. Therefore, we also adopt SD resolution for our dataset. This choice reduces storage requirements and facilitates direct comparison with the BOP PBR images, which also use SD resolution. Additionally, we investigate event data representations suitable for processing with Convolutional Neural Networks (CNNs). While less computationally efficient than Spiking Neural Networks (SNNs), CNNs currently offer superior inference speeds [ 10 ] . Based on the above, our key contributions are; 1. A large-scale synthetic event dataset for object pose estimation, and 2. a comparative analysis of event representations for CNN-based inference. This paper is structured as follows: Section 2 details our synthetic data generation pipeline. Section 3 discusses and evaluates different event data representations for CNN processing, while Section 4 explains the structure and storage of the captured data. We conclude with Section 5 , which summarizes our results, discusses the limitations, and outlines potential future work. 2 Synthetic event-data generation Our synthetic data generation follows the BOP20 methodology [ 6 ] , producing a dataset of 50,000 distinct camera views using BlenderProc [ 2 ] . The views are organized into sequences of 25, with each sequence capturing the same object arrangement from a set of random camera poses. To synthesize event data corresponding to each camera pose, we introduce controlled camera motion, ensuring event generation across both foreground objects and background elements. For each target pose, we define a linear motion trajectory originating from a randomly sampled point on a sphere surrounding the target position and extending linearly toward it. We simulate this motion in BlenderProc over a 34 ms duration, rendering images at 1000 fps with a constant camera velocity of 1 cm/ms. The resulting high-frame-rate image sequences are subsequently converted into event streams using the IEBCS event camera simulator [ 8 ] . The event sequences generated through this process are sufficiently long to be accumulated into histograms at 30 fps (33 ms per frame), enabling direct comparison with conventional RGB video streams. This accumulation strategy facilitates fair benchmarking against standard frame-based approaches. The complete generation pipeline required rendering approximately 1.7 million high-quality PBR frames, equivalent to 28 minutes and 20 seconds of video at the simulated 1000 fps rate. This computationally intensive process required approximately one week utilizing six modern GPUs. 3 Comparing event representations for CNN inference Although Spiking Neural Networks (SNNs) offer computational advantages for processing event data, recent studies indicate that conventional CNN architectures still achieve superior frame rates [ 10 ] . Consequently, we evaluate various event representations [ 3 ] for their compatibility with CNN-based processing. Figure 2 : Common event representations: (left) event histogram and (right) time-surface We begin with event histograms and time-surfaces, as these map naturally to 2D grid structures familiar from image processing, making them widely adopted in literature [ 7 ] (see Figure 2 ). Figure 3 : Time-surface representations with polarity encoding: (left) 2 channels in blue-green (right) 1 channel (black = off, white = on) Time-surfaces typically employ an exponential decay function. We formulate this as: I(\mathbf{x},t)=255\cdot exp(-4\cdot\dfrac{t_{max}-t}{t_{max}}) (1) where t_{\text{max}}=33 ms represents the maximum time delta in the time-surface. The decay

Read the complete text