EventVGGT: Exploring Cross-Modal Distillation for Consistent Event-based Depth Estimation
Yinrui Ren, Jinjing Zhu, Kanghao Chen, Zhuoxiao Li, Jing Ou, Zidong Cao, Tongyan Hua, Peilun Shi, Yingchun Fu, Wufan Zhao, Hui Xiong HKUST(GZ) CUHK SCNU · 2026
Read EventVGGT: Exploring Cross-Modal Distillation for Consistent Event-based Depth Estimation by Yinrui Ren, Jinjing Zhu, Kanghao Chen, Zhuoxiao Li, Jing Ou, Zidong Cao, Tongyan Hua, Peilun Shi, Yingchun Fu, Wufan Zhao, Hui Xiong HKUST(GZ) CUHK SCNU (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-09385.txt.
Introduction
Yinrui Ren and colleagues present EventVGGT, a framework for event-based monocular depth estimation that treats the event stream as a coherent video rather than lone frames. Dense depth labels are scarce, and past annotation-free methods distilled knowledge from vision foundation models frame by frame, losing temporal consistency. EventVGGT pulls spatio-temporal, multi-view priors from the Visual Geometry Grounded Transformer into the event domain. A tri-level strategy, starting with cross-modal feature mixing, bridges the modality gap.
Catalog record
| CREATOR | Yinrui Ren, Jinjing Zhu, Kanghao Chen, Zhuoxiao Li, Jing Ou, Zidong Cao, Tongyan Hua, Peilun Shi, Yingchun Fu, Wufan Zhao, Hui Xiong HKUST(GZ) CUHK SCNU |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 40245 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.09385
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
EventVGGT: Exploring Cross-Modal Distillation for Consistent Event-based Depth Estimation Yinrui Ren, Jinjing Zhu, Kanghao Chen, Zhuoxiao Li, Jing Ou, Zidong Cao, Tongyan Hua, Peilun Shi, Yingchun Fu, Wufan Zhao, Hui Xiong HKUST(GZ) CUHK SCNU Abstract Event cameras offer superior sensitivity to high-speed motion and extreme lighting, making event-based monocular depth estimation a promising approach for robust 3D perception in challenging conditions. However, progress is severely hindered by the scarcity of dense depth annotations. While recent annotation-free approaches mitigate this by distilling knowledge from Vision Foundation Models (VFMs), a critical limitation persists: they process event streams as independent frames. By neglecting the inherent temporal continuity of event data, these methods fail to leverage the rich temporal priors encoded in VFMs, ultimately yielding temporally inconsistent and less accurate depth predictions. To address this, we introduce EventVGGT , a novel framework that explicitly models the event stream as a coherent video sequence. To the best of our knowledge, we are the first to distill spatio-temporal and multi-view geometric priors from the Visual Geometry Grounded Transformer (VGGT) into the event domain. We achieve this via a comprehensive tri-level distillation strategy: (i) Cross-Modal Feature Mixture (CMFM) bridges the modality gap at the output level by fusing RGB and event features to generate auxiliary depth predictions; (ii) Spatio-Temporal Feature Distillation (STFD) distills VGGT’s powerful spatio-temporal representations at the feature level; and (iii) Temporal Consistency Distillation (TCD) enforces cross-frame coherence at the temporal level by aligning inter-frame depth changes. Extensive experiments demonstrate that EventVGGT consistently outperforms existing methods—reducing the absolute mean depth error at 30m by over 53% on EventScape (from 2.30 to 1.06)—while exhibiting robust zero-shot generalization on the unseen DENSE and MVSEC datasets. 1 Introduction Event cameras 9 ; 43 are bio-inspired sensors that asynchronously encode logarithmic intensity changes into sparse event streams. Unlike conventional RGB cameras, they offer exceptional temporal resolution and dynamic range, enabling robust perception under high-speed motion and challenging illumination conditions 25 ; 37 . Consequently, they have driven significant advancements across various vision tasks, including object detection 24 ; 29 , tracking 6 ; 10 ; 42 , and segmentation 19 ; 20 ; 7 ; 22 . Among these, event-based monocular depth estimation 16 ; 8 ; 28 has emerged as a critical capability for autonomous driving and robotic navigation 38 ; 31 . However, despite recent efforts to fuse the complementary strengths of event and RGB modalities 28 ; 14 , progress in event-based depth estimation remains bottlenecked by the scarcity of large-scale datasets with dense depth annotations. To address this scarcity, recent methods 48 ; 2 leverage Vision Foundation Models (VFMs) to generate high-quality pseudo-depth labels from paired RGB-event data, bypassing the need for ground-truth annotations. For instance, EventDAM 48 employs dense-to-sparse distillation on isolated event frames, while DepthAnyEvent 2 integrates temporal modules before transferring knowledge from a single-image teacher (DAM V2 41 ). Although these VFM-based approaches alleviate the annotation bottleneck, a critical limitation persists: by treating event streams as independent frames, they fail to leverage the inherent temporal continuity of the data and the temporal priors of VFMs, ultimately yielding temporally inconsistent and suboptimal depth predictions . Among available VFMs, models equipped with multi-view geometric reasoning are ideal teachers, as their spatio-temporal 3D priors naturally align with the continuous dynamics of event streams. A prime example is the Visual Geometry Grounded Transformer (VGGT) 39 , which jointly infers depth, camera poses, and point clouds from multiple views. Unlike single-image baselines ( e.g . , DAM V2 41 ) that lack cross-frame awareness, VGGT implicitly enforces geometric consistency across views via alternating frame-wise and global attention. Inspired by this, we propose to adapt VGGT’s powerful multi-view reasoning for consistent event-based depth estimation. By treating the asynchronous event stream as a dense sequence of views, we explicitly exploit VGGT to model the temporal constraints and geometric coherence inherent in event data. To this end, we introduce EventVGGT , an annotation-free framework that treats the event stream as a continuous video sequence and distills rich spatio-temporal priors from a VGGT teacher to achieve consistent event-based depth estimation (see Fig. ). To effectively transfer this knowledge, we propose a tri-level distillation strategy: (i) At the output level , we introduce the Cross-Modal Feature Mixture (CMFM) module, which bridges the substantial modality gap by fusing RGB and event features to generate an auxiliary depth prediction. By explicitly supervising this auxiliary output with the teacher’s high-fidelity RGB depth maps, we effectively distill the teacher’s geometric priors into the event-based student. (ii) At the feature level , to fully exploit the teacher’s internal representations, we propose Spatio-Temporal Feature Distillation (STFD) , aligning the student’s intermediate features with those of VGGT to enforce the learning of spatial structure and temporal correspondence. (iii) At the temporal level , we introduce a Temporal Consistency Distillation (TCD) loss to supervise the inter-frame changes of consecutive depth maps rather than their absolute values. By penalizing discrepancies in inter-frame depth changes, TCD compels the student to inherit the teacher’s geometrically coherent temporal flow, yielding accurate and stable depth sequences. We validate EventVGGT on EventScape 12 and MVSEC 44 datasets, achieving new state-of-the-art performance against both event- and image-based methods. Furthermore, we demonstrate its robust zero-shot generalization: when trained solely on EventScape and evaluated on the unseen DENSE 16 and MVSEC datasets, EventVGGT maintains a substantial performance margin. The versatility of our approach is further highlighted by its effective extension to distilling VGGT for other geometric tasks, including camera pose and point map estimation from events. In summary, our main contributions are three-fold: • We introduce EventVGGT, the first framework to distill spatio-temporal priors from a multi-view foundation model (VGGT) into an event-based student, enabling temporally consistent, annotation-free depth estimation. • We propose a comprehensive tri-level distillation strategy: Cross-Modal Feature Mixture (CMFM), Spatio-Temporal Feature Distillation (STFD), and Temporal Consistency Distillation (TCD). • EventVGGT achieves state-of-the-art results on EventScape and MVSEC, demonstrates strong zero-shot generalization on DENSE, and seamlessly extends to camera pose and point cloud estimation. 2 Related Work Event-based monocular depth estimation. Traditional RGB-based monocular depth estimation 4 ; 3 ; 41 is constrained by fixed frame rates and degrades severely under challenging illumination. To overcome these limitations, event-based depth estimation has emerged as a robust alternative, enabling high-frequency predictions in dynamic lighting and rapid motion scenarios 43 ; 35 . Early seminal works, such as E2Depth 16 , leverage recurrent convolutional networks to generate dense depth maps from event streams. Subsequent methods 11 ; 34 ; 8 ; 28 integrate paired RGB and event data to further enhance accuracy. Despite these advancements, most supervised approaches remain bottlenecked by their reliance on expensive, extensively annotated datasets. To bypass this, recent annotation-free works like DepthAnyEvent 2 and EventDAM 48 utilize cross-modal distillation to transfer knowledge from image-based Vision Foundation Models (VFMs). However, by treating event streams as isolated frames, these methods fail to adequately exploit the inherent temporal dynamics of events and the temporal priors of VFMs. To address this, our work explicitly models the event stream as a continuous video sequence, ensuring that the estimated depth maps maintain strict geometric consistency throughout the sequence . Cross-modal knowledge distillation. Knowledge distillation (KD) classically transfers learned representations from a high-capacity