INHerit-SG: Incremental Hierarchical Semantic Scene Graphs with RAG-Style Retrieval

YukTungSamuel Fang 1 , Zhikang Shi 1 , Jiabin Qiu 1 , Zixuan Chen 1 , Jieqi Shi 1* , Hao Xu 2 , Jing Huo 1* , Yang Gao 1 · 2026

Read INHerit-SG: Incremental Hierarchical Semantic Scene Graphs with RAG-Style Retrieval by YukTungSamuel Fang 1 , Zhikang Shi 1 , Jiabin Qiu 1 , Zixuan Chen 1 , Jieqi Shi 1* , Hao Xu 2 , Jing Huo 1* , Yang Gao 1 (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-12971.txt.

Description

Finding objects in 3D scenes from queries that mix hierarchy, attributes, negation, and chained spatial relations is hard for existing maps and memories. YukTungSamuel Fang and seven co-authors propose INHerit-SG, a layered semantic scene graph that grows over time, with retrieval to match. Geometric and semantic processing run asynchronously, and event-driven maintenance organises observations into a hierarchy expressed in plain text. That same memory supports direct reading by strong hosted models or parsed structured retrieval for smaller local models.

Direct reading wins with powerful models, while structured retrieval helps smaller models and cuts per-query API cost for hosted ones. Tests on 6,084 queries across 36 scenes plus three real environments show INHerit-SG beating the strongest baselines, with an overall success lead of 9.8 points on real robots and the largest gains on attribute and compositional queries.

Catalog record

CREATORYukTungSamuel Fang 1 , Zhikang Shi 1 , Jiabin Qiu 1 , Zixuan Chen 1 , Jieqi Shi 1* , Hao Xu 2 , Jing Huo 1* , Yang Gao 1
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE73 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602-12971

Browse the full library · About the Tower of Babel