Ego4OOD: Rethinking Egocentric Video Domain Generalization via Covariate Shift Scoring

Zahra Vaseqi, James Clark · 2026

Read Ego4OOD: Rethinking Egocentric Video Domain Generalization via Covariate Shift Scoring by Zahra Vaseqi, James Clark (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-17056.txt.

Introduction

Zahra Vaseqi and James Clark rethink tests for head-cam video tools. Head-cam shots show the world from the wearer's view. Tools must name acts in those shots in new homes. New homes shift light, gear, and scene in big ways. Past tests mixed two kinds of shift at once. Their 2026 preprint splits those shifts with a score rule.

Their rule, Ego4OOD, scores how far a test set has shifted. Input shifts split clean from sense shifts. Clean splits let teams grade true spread skill. Such grades matter for aid tools for the blind. They also matter for work and sport head cams.

Catalog record

CREATORZahra Vaseqi, James Clark
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE31 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2601-17056

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Ego4OOD: Rethinking Egocentric Video Domain Generalization via Covariate Shift Scoring Zahra Vaseqi, James Clark Abstract Egocentric video action recognition under domain shifts remains challenging due to large intra-class spatio-temporal variability, long-tailed feature distributions, and strong correlations between actions and environments. Existing benchmarks for egocentric domain generalization often conflate covariate shifts with concept shifts, making it difficult to reliably evaluate a model’s ability to generalize across input distributions. To address this limitation, we introduce Ego4OOD, a domain generalization benchmark derived from Ego4D that emphasizes measurable covariate diversity while reducing concept shift through semantically coherent, moment-level action categories. Ego4OOD spans eight geographically distinct domains and is accompanied by a clustering-based covariate shift metric that provides a quantitative proxy for domain difficulty. We further leverage a one-vs-all binary training objective that decomposes multi-class action recognition into independent binary classification tasks. This formulation is particularly well-suited for covariate shift by reducing interference between visually similar classes under feature distribution shift. Using this formulation, we show that a lightweight two-layer fully connected network achieves performance competitive with state-of-the-art egocentric domain generalization methods on both Argo1M and Ego4OOD, despite using fewer parameters and no additional modalities. Our empirical analysis demonstrates a clear relationship between measured covariate shift and recognition performance, highlighting the importance of controlled benchmarks and quantitative domain characterization for studying out-of-distribution generalization in egocentric video. 1 Introduction Independent and identically distributed (i.i.d.) data is a key assumption in many research problems. This assumption is often violated across training and test data in practice. Image and video classification models are prone to learn spurious correlations in visual content; this compromises their ability to generalize when a distributional shift between train and test sets is present. Such data biases are inherent in real-world computer vision datasets due to capture bias and geographical constraints [ 1 ] in data collection. When a strong correlation between two features in the training set exists, and the test set does not have the same type of correlation, the generalization performance suffers [ 2 ] . The issue with data distributional shifts has been widely studied in the context of synthetic shifts in image classification; however, it is unclear whether the synthetic robustness methods transfer to natural shifts [ 3 , 4 , 5 ] . While training on incrementally larger and more diverse datasets [ 4 ] with data augmentation [ 6 , 2 ] have been shown to improve robustness in presence of distributional shifts; we still lack a comprehensive understanding of the underlying factors in model training that leads to performance discrepancies across domains. The notion of in-distribution in video understanding is inherently broad, encompassing a wide range of spatio-temporal variations observed during training. These include changes in motion dynamics, illumination (e.g., overcast or nighttime scenes), background clutter, and camera viewpoints that preserve the underlying semantic task. At the same time, shifts along these same factors—such as transitions between indoor and outdoor environments, extreme lighting conditions, or novel viewpoints—may constitute out-of-distribution data when they fall outside the support of the training distribution. As a result, out-of-distribution in videos is not defined by the presence of a specific factor, but by whether that factor, or its interaction with others over time, is sufficiently represented during training. The image domain generalization benchmarks, where out-of-distribution shifts are typically defined through explicit domain splits targeting specific nuisance factors, such as image style [ 7 , 8 , 9 ] , color [ 10 ] , rotation [ 11 ] , or background [ 12 ] . In videos, the distinction between in-distribution and out-of-distribution data is often less clear. This issue is further amplified in egocentric video, where the data distribution is strongly conditioned on the camera wearer’s behavior and environment. Non-stationary camera motion, rapidly varying viewpoints, and task-driven interactions induce large intra-class spatio-temporal variability, while user-specific and long-tailed activity distributions imply that the training data covers only a sparse subset of possible factor combinations. Consequently, egocentric video substantially blurs the distinction between in-distribution variability and out-of-distribution data, making robust generalization particularly challenging. In this work, we investigate domain generalization in egocentric action recognition. We find that even a simple two-layer fully connected network can achieve performance on par with current state-of-the-art methods for domain generalization on egocentric action recognition. To support further research, we introduce Ego4OOD, a domain generalization benchmark built on the existing Ego4D dataset that reorganizes and categorizes videos to evaluate out-of-distribution generalization. Furthermore, we propose a covariate shift metric that provides a quantitative score for each domain, enabling systematic analysis of how visual features vary across domains. Our qualitative observations of visual differences across domains are consistent with the quantitative covariate shift scores, highlighting a strong correlation between perceived domain variation and the metric. The contributions of this work are threefold: (i) we highlight the effectiveness of a simple network architecture for domain generalization in egocentric action recognition, (ii) we introduce Ego4OOD, a benchmark for systematic evaluation of out-of-distribution generalization in egocentric videos, and (iii) we provide an accompanying covariate shift metric and analysis that captures inter-domain feature variations. In the following sections, we first review related work on domain generalization and egocentric video analysis, then describe the construction of the Ego4OOD benchmark and our experimental setup, and finally present results and analysis that illustrate the challenges and insights for generalization across diverse video domains. 2 Related Work 2.1 Egocentric Action Recognition In traditional third-person vision, a fixed camera passively observes the scene, whereas in egocentric vision the camera is worn by the actor, recording firsthand interactions with the surrounding environment. Research in egocentric vision has grown rapidly in recent years, driven by the release of large-scale benchmarks such as Epic-Kitchens [ 13 ] , Ego4D [ 14 ] , Charades-Ego [ 15 ] . In egocentric videos, the environment often occupies a large portion of the field of view, and rapidly changing viewpoints make it challenging to recognize actions based solely on scene context or motion trajectories, which are more informative and stable in third-person videos [ 16 , 17 ] . This highlights the importance of characterizing actions independently of the environment and generalizing to new domains [ 18 ] . Ego-Topo [ 19 ] introduces the notion of affordances in egocentric vision, modeling actions as dependent on the environment and objects present—for example, a kitchen sink affording washing. EgoZAR [ 18 ] highlights a key failure mode of this approach by modeling environmental affordances as activity-centric zones derived from co-occurrence biases. While domain-specific affordances can improve performance when actions are evaluated in previously seen environments, it negatively impacts generalization to unseen environments. EgoZAR [ 18 ] suggests an adversarial classifier to extract domain-agnostic zone psuedo labels from the motion features. On the other hand, Plizzari et al. propose Cross-Instance Reconstruction (CIR) [ 20 ] , which represents an action as a weighted combination of video instances drawn from other domains. CIR encourages domain-agnostic representations by reconstructing each video at the feature level using a support set of samples from other domains within the same batch. The reconstructed representation is then aligned with the corresponding textual narration via a contrastive video–text objective. By learning to reconstruct the same action from samples from other domains, CIR promotes

Read the complete text