Cognitive Synergy Architecture: SEGO for Human-Centric Collaborative Robots
Jaehong Oh · 2025
Read Cognitive Synergy Architecture: SEGO for Human-Centric Collaborative Robots by Jaehong Oh (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-13149.txt.
Introduction
Jaehong Oh presents a robot thinking system called SEGO in a 2025 paper. SEGO stands for Semantic Graph Ontology, a mapping design for robots that work beside people. A helpful partner robot must join raw geometry with meaning and with reasons it can state.
The system builds live cognitive scene graphs of the room and its objects. Those graphs hold not just positions, but semantic links and logical consistency too. It fuses location tracking from SLAM with deep-learning object spotting and ontology-based reasoning. The robot can then map in real time and explain its map in human terms. Shared understanding is the goal for safe teamwork.
Catalog record
| CREATOR | Jaehong Oh |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 33 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2506-13149
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Cognitive Synergy Architecture: SEGO for Human-Centric Collaborative Robots Jaehong Oh Abstract This paper presents SEGO (Semantic Graph Ontology), a cognitive mapping architecture designed to integrate geometric perception, semantic reasoning, and explanation generation into a unified framework for human-centric collaborative robotics. SEGO constructs dynamic cognitive scene graphs that represent not only the spatial configuration of the environment but also the semantic relations and ontological consistency among detected objects. The architecture seamlessly combines SLAM-based localization, deep-learning-based object detection and tracking, and ontology-driven reasoning to enable real-time, semantically coherent mapping. A systematic experimental evaluation was conducted using the TUM RGB-D dataset, with frame rates ranging from 10 to 60 frames per second (FPS). Results demonstrated that SEGO achieves significant improvements in semantic mapping quality up to 30 FPS, with the Semantic Recognition Quality Index (SRQI) increasing from 0.662 at 10 FPS to 0.703 at 30 FPS, beyond which gains plateau. This frame-rate-dependent behavior aligns with known limits of human perceptual integration, supporting SEGO’s suitability for intuitive human-robot interaction. Moreover, SEGO’s reasoning traceability enables transparent and interpretable decision-making, fostering trust and predictability in collaborative settings. The study introduces novel metrics, including SRQI, violation rate, and relation entropy, to quantitatively assess semantic mapping performance. The results validate SEGO’s frame-rate-aware design and its capacity to deliver cognitively transparent mapping with computational efficiency. The architecture provides a principled foundation for future cognitive robotic systems requiring real-time semantic understanding, logical consistency, and explainable reasoning in complex, dynamic environments. I Introduction Robotic systems designed for autonomous operation have demonstrated significant advances in perception, localization, and geometric mapping. Techniques such as simultaneous localization and mapping (SLAM), 3D reconstruction, and object detection have enabled robots to navigate and interpret their environments with increasing accuracy. However, these advancements remain predominantly confined to geometric representations, offering little in terms of semantic understanding or relational reasoning. In collaborative human-robot environments—where contextual awareness, shared understanding, and explainability are paramount—this geometric focus proves insufficient, limiting the robot’s ability to act as a true partner in complex tasks. Recent surveys, including our prior review on cognitive collaborative robots [ 1 ] , have underscored the urgent need for robotic frameworks that transcend geometric mapping by integrating semantic perception, ontological reasoning, and explainable control. While isolated efforts in semantic SLAM and knowledge-based scene representation have emerged, they typically lack cohesive architectures that unify geometric, semantic, and logical layers into a single cognitive mapping system suitable for human-centric cooperation. In response to this gap, we propose SEGO (Semantic Graph Ontology mapper) , a novel architecture designed to provide robots with the ability to construct semantic-level cognitive maps. SEGO generates cognitive scene graphs that encode not only spatial coordinates and object identities but also semantic relations (e.g., left_of , above , inside ) and ontological constraints derived from domain knowledge. Each node and edge in the graph is enriched with logical consistency checks, ensuring that the internal world model is both geometrically sound and semantically coherent. The SEGO architecture is characterized by three core design objectives: • Ontological Integration: SEGO incorporates domain-specific ontologies that define object categories, permissible relations, and hierarchical structures. This allows the system to reason about the world in alignment with human-understandable concepts. • Semantic Consistency: The framework actively monitors and minimizes logical violations, detecting contradictions such as spatial impossibilities or relation inconsistencies within the scene graph. • Explainable Mapping: SEGO produces interpretable outputs where semantic relations and object associations can be traced back to perceptual data and reasoning chains, supporting transparency in robot decision-making. A distinctive feature of SEGO is its focus on the temporal dynamics of semantic perception. Although frame rate (FPS) has been extensively studied in geometric SLAM, its impact on semantic mapping quality remains largely unexplored. Given that human visual cognition typically operates optimally at 24–30 FPS, we hypothesize that a robot’s semantic mapping capability may similarly exhibit frame rate dependency, with potential saturation effects beyond certain thresholds. To quantitatively evaluate SEGO’s semantic mapping performance, we introduce the Semantic Recognition Quality Index (SRQI) —a composite metric that captures semantic consistency, relational entropy, and logical coherence of generated scene graphs. Through rigorous experimentation using the TUM RGB-D dataset, we assess SEGO under varying FPS conditions (10, 15, 20, 30, and 60 FPS) and analyze its performance in terms of SRQI, semantic violation rates, relation entropy, and structural complexity of the cognitive scene graphs. The primary contributions of this work are as follows: 1. SEGO Architecture: We introduce SEGO, a unified semantic mapping architecture that combines ontological reasoning, logical validation, and cognitive scene graph construction. 2. Quality Metrics: We propose SRQI and associated metrics for assessing semantic mapping quality from both logical and spatial perspectives. 3. Experimental Analysis: We conduct extensive experiments to study the impact of FPS on semantic mapping performance and identify frame rate saturation phenomena. 4. Alignment with Human Cognition: We provide insights into how SEGO’s semantic mapping aligns with human perceptual rhythms and supports explainable, collaborative robotics. This work builds on the vision articulated in our previous review [ 1 ] , operationalizing the integration of semantic-level mapping and explainable control into a concrete framework for cognitive robotics. II Background and Related Work II-A SLAM and Semantic SLAM Simultaneous localization and mapping (SLAM) has long served as a cornerstone in autonomous robotics, enabling robots to construct geometric representations of unknown environments while localizing themselves within these maps. Classical SLAM systems solve the joint estimation problem of robot pose and map features by minimizing a cost function of the form: \mathcal{L}(X,M)=\sum_{i}\|z_{i}-h(x_{i},m_{i})\|^{2} (1) where X=\{x_{i}\} denotes the robot trajectory, M=\{m_{i}\} the map landmarks, z_{i} the observations, and h(\cdot) the observation model. Among geometric SLAM systems, ORB-SLAM2 [ 2 ] represents one of the most influential works. It employs ORB features for visual tracking, loop closure detection via bag-of-words place recognition, and pose graph optimization through bundle adjustment. ORB-SLAM2 delivers precise, real-time 6-DoF camera pose estimates \mathbf{T}_{t}\in SE(3) and sparse map point clouds suitable for navigation and mapping. Despite these successes, traditional SLAM constructs purely metric maps devoid of semantic understanding. This limitation prevents SLAM from supporting higher-level tasks requiring context awareness, symbolic reasoning, or human-centric collaboration. Semantic SLAM augments geometric SLAM with semantic labels, enabling the robot to associate map elements with object categories, instances, or properties. For example, SemanticFusion [ 3 ] combines ElasticFusion’s surfel-based dense mapping with per-frame semantic segmentation using convolutional neural networks (CNNs). It fuses pixel-wise semantic predictions over time into a dense 3D semantic map: P(c|s)=\frac{1}{N}\sum_{t=1}^{N}P_{t}(c|s) (2) where P(c|s) is the class probability of surfel s , averaged over N observations. While semantic SLAM represents progress toward contextual mapping, its semantic annotations are largely local and geometric, lacking relational reasoning. These systems primarily label what is present rather than modeling how entities relate within a scene. II-B Scene Graphs in Robotics Scene graphs formalize structured knowledge as \mathcal{G}=(V,E) , where V represents detected objects and E encodes pairwise relations: E=\{(v_{i},r_{ij},v_{j})\;|\;v_{i},v_{j}\in V,r_{ij}\in\mathcal{R}\} (3) This structure enables querying, reasoning, and decision-making. In robotics, scene graphs bridge raw sensory data and symbolic reasoning. They support manipulation, navigation, and human-robot interaction by encoding contextual relations such as left_of , on_top_of , or inside . Existing frameworks often rely on static scenes or pre-mapped environments, with limited dynamic integration. II-C Ontology-Based Reasoning in