Enhancing Egocentric Object Detection in Static Environments using Graph-based Spatial Anomaly Detection and Correction
Anonymous Submission, Vishakha Lall, Yisi Liu · 2025
Read Enhancing Egocentric Object Detection in Static Environments using Graph-based Spatial Anomaly Detection and Correction by Anonymous Submission, Vishakha Lall, Yisi Liu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-07624.txt.
Introduction
Vishakha Lall and Yisi Liu fix detection errors in first-person views in twenty twenty-five. In static settings, objects keep a fixed spatial layout across scenes. Top detectors ignore that prior, so they miss objects or mislabel them. Cluttered and occluded views make the errors worse.
Their pipeline models spatial relations between objects as a graph. A graph neural network trained on hand-labeled data spots labels that cannot be right. It then predicts corrected labels from the neighborhood around each object. The method works alone or as a post-processing step for detectors such as YOLOv7 and RT-DETR. Readers get a way to let a room's stable layout police a detector's mistakes.
Catalog record
| CREATOR | Anonymous Submission, Vishakha Lall, Yisi Liu |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 21 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2508-07624
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Enhancing Egocentric Object Detection in Static Environments using Graph-based Spatial Anomaly Detection and Correction Thanks: Thanks to Singapore Maritime Institute (SMI) Anonymous Submission, Vishakha Lall, Yisi Liu Abstract In many real-world applications involving static environments, the spatial layout of objects remains consistent across instances. However, state-of-the-art object detection models often fail to leverage this spatial prior, resulting in inconsistent predictions, missed detections, or misclassifications, particularly in cluttered or occluded scenes. In this work, we propose a graph-based post-processing pipeline that explicitly models the spatial relationships between objects to correct detection anomalies in egocentric frames. Using a graph neural network (GNN) trained on manually annotated data, our model identifies invalid object class labels and predicts corrected class labels based on their neighbourhood context. We evaluate our approach both as a standalone anomaly detection and correction framework and as a post-processing module for standard object detectors such as YOLOv7 and RT-DETR. Experiments demonstrate that incorporating this spatial reasoning significantly improves detection performance, with mAP@50 gains of up to 4%. This method highlights the potential of leveraging the environment’s spatial structure to improve reliability in object detection systems. I Introduction High-stakes, high-complexity domains such as maritime navigation demand rigorous validation of situational awareness and correct use of critical systems. As maritime training moves toward immersive, high-fidelity simulators, there is a growing need for automated systems that can analyse how users interact with onboard instruments and detect deviations from standard operational behaviour. However, egocentric video captured in these environments presents challenges for conventional computer vision pipelines due to occlusions, varying perspectives, and cluttered layouts. Egocentric vision has emerged as a rich source of contextual and behavioural cues in fields ranging from robotics to human-computer interaction. Prior work has used wearable cameras to model attention, track tasks, or understand indoor navigation [ 1 , 2 ] . However, few studies address static but semantically dense environments, such as ship bridges, where spatial understanding is crucial and traditional appearance-based models may fail due to perspective distortion or partial views. This challenge, and the opportunity it presents, is not unique to the maritime domain. In any static environment, object detection performance can be significantly enhanced by incorporating spatial relationships among scene elements, allowing models to reason beyond isolated detections. Graphs provide a powerful abstraction for modelling relationships between objects in visual scenes. Scene graphs have been widely used in image captioning [ 3 ] , visual question answering [ 4 ] , and robotics [ 5 ] . In structured indoor environments, spatial graphs have shown promise in representing room layouts or object co-occurrences [ 6 ] . Our work extends this idea to static environments, explicitly modelling spatial displacements, angles, overlap and object sizes as edge features for downstream anomaly detection and correction of object detection predictions. In this paper, we propose a graph-based framework for modelling the spatial layout of static environments, such as that of a maritime simulator using egocentric video. By treating each annotated object as a node in a spatial graph, we learn the structural relationships between them and use this information for two downstream tasks: anomaly detection, which identifies mislabeled objects in new scenes, and anomaly correction, which proposes corrected labels based on spatial context. Our approach is evaluated on a dataset collected in a maritime bridge simulator, where egocentric views from a moving participant were manually annotated to create spatially rich scene representations. We simulate annotation errors to train and evaluate a multi-task Graph Neural Network (GNN) that jointly classifies node validity and predicts corrected labels. Additionally, we demonstrate that applying the anomaly correction as a postprocessing step significantly improves the performance of state-of-the-art object detectors such as YOLOv7 and RT-DETR trained on the same dataset. In summary, our contributions are: 1. A spatial graph-based representation of egocentric maritime simulator scenes, capturing both semantic and geometric object relationships. 2. A multi-task GNN model for node-level anomaly detection and correction, trained using synthetically corrupted scenes. 3. An empirical analysis of how structural correction improves object detection accuracy in noisy but static environments. II Dataset The dataset was collected at the Advanced Navigation Research Simulator (ANRS), located at the Centre of Excellence in Maritime Safety (CEMS), Singapore. Data acquisition was conducted using the scene camera of Tobii Pro Glasses 3, worn by a participant performing standardised navigational tasks. This setup enabled the collection of egocentric video sequences as the participant moved within the simulator environment, offering varied perspectives of a static but spatially complex ship bridge layout. The total number of frames extracted from the videos were 26k. To construct a spatially-aware visual representation of the simulator, key panels and equipment were manually annotated in each frame using bounding boxes and semantic labels for 39 distinct classes representing equipment and panels in the ship bridge simulator. Each annotated object includes its label, bounding box coordinates, geometric centre, and spatial size. These annotations serve as the basis for building a graph representation of the environment. For each annotated frame, a graph is constructed where nodes correspond to objects, and node features encode the object’s label, spatial centre, and size. Edges are formed by connecting each node to its k nearest neighbours based on the Euclidean distance between object centres. Edge features capture pairwise spatial relations, including relative displacement, Euclidean distance, and angular orientation, Intersection over Union (IoU) of bounding boxes, and the relative size ratio between objects. Figure 1 presents examples of the generated graphs overlaid on egocentric images, illustrating the spatial relationships between annotated panels and equipment from the participant’s visual perspective. Each node v_{i} represents an object (panel/equipment) in the simulator, with the following feature vector: \mathbf{f_{i}}=[label_{i},x_{i},y_{i},w_{i},h_{i}] (1) where label_{i} is the object’s semantic label, (x_{i},y_{i}) are the coordinates of the object’s centre, (w_{i},h_{i}) are the width and height of the object’s bounding box. For each edge e_{ij} between nodes v_{i} and v_{j} , the feature vector is: \mathbf{e_{ij}}=[\Delta x_{ij},\Delta y_{ij},d_{ij},\theta_{ij},IoU_{ij},\rho_{ij}] (2) where, (\Delta x_{ij},\Delta y_{ij}) is the horizontal x_{j}-x_{i} and vertical y_{j}-y_{i} displacement respectively, d_{ij} is the Euclidean distance \sqrt{(x_{j}-x_{i})^{2}+(y_{j}-y_{i})^{2}} , \theta_{ij} is the angle in degrees between v_{i} and v_{j} measured as arctan2(y_{j}-y_{i},x_{j}-x_{i})\frac{180}{\pi} , IoU_{ij} is the Intersection over Union of the bounding boxes for v_{i} and v_{j} measured as \frac{A_{i}\cap A_{j}}{A_{i}\cup A_{j}} where A_{i},A_{j} are the areas of v_{i} and v_{j} respectively, and \rho_{ij} is the relative size ratio between v_{i} and v_{j} measured as \frac{w_{j}h_{j}}{w_{i}h_{i}} . (a) Centre perspective (b) Left perspective (c) Right perspective (d) Top right perspective Fig. 1 : Samples from the dataset showing annotated graph nodes with green bounding boxes, node labels in white text, and spatial edges to neighbouring nodes highlighted in cyan. In addition to supporting graph construction, the annotated dataset is also used to train object detection models, which serve as a validation mechanism for the anomaly detection and correction algorithm. This enables a systematic evaluation of the improvement of object detection accuracy using graph-based anomaly corrections. III Proposed Methodology III-A Data Augmentation To train and evaluate the robustness of anomaly detection and correction models, we augment the original dataset with synthetic negative samples. These modified samples simulate realistic deviations from the expected spatial configuration. Specifically: 1. A random subset of objects (up to \rho per frame) are selected. 2.