INHerit-SG: Incremental Hierarchical Semantic Scene Graphs with RAG-Style Retrieval
YukTungSamuel Fang 1 , Zhikang Shi 1 , Jiabin Qiu 1 , Zixuan Chen 1 , Jieqi Shi 1* , Hao Xu 2 , Jing Huo 1* , Yang Gao 1 · 2026
Read INHerit-SG: Incremental Hierarchical Semantic Scene Graphs with RAG-Style Retrieval by YukTungSamuel Fang 1 , Zhikang Shi 1 , Jiabin Qiu 1 , Zixuan Chen 1 , Jieqi Shi 1* , Hao Xu 2 , Jing Huo 1* , Yang Gao 1 (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-12971.txt.
Description
Finding objects in 3D scenes from queries that mix hierarchy, attributes, negation, and chained spatial relations is hard for existing maps and memories. YukTungSamuel Fang and seven co-authors propose INHerit-SG, a layered semantic scene graph that grows over time, with retrieval to match. Geometric and semantic processing run asynchronously, and event-driven maintenance organises observations into a hierarchy expressed in plain text. That same memory supports direct reading by strong hosted models or parsed structured retrieval for smaller local models.
Direct reading wins with powerful models, while structured retrieval helps smaller models and cuts per-query API cost for hosted ones. Tests on 6,084 queries across 36 scenes plus three real environments show INHerit-SG beating the strongest baselines, with an overall success lead of 9.8 points on real robots and the largest gains on attribute and compositional queries.
Catalog record
| CREATOR | YukTungSamuel Fang 1 , Zhikang Shi 1 , Jiabin Qiu 1 , Zixuan Chen 1 , Jieqi Shi 1* , Hao Xu 2 , Jing Huo 1* , Yang Gao 1 |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 73 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2602-12971