SIR: Structured Image Representations for Explainable Robot Learning
Paul Mattes, Jan Schwab, Jens Bosch, Nils Blank, Maximilian Xiling Li, Minh-Trung Tang, Moritz Haberland, Rudolf Lioutikov, [0.5em] Intuitive Robots Lab, Karlsruhe Institute of Technology, Germany Robotics Institute Germany[0.3em] {paul.mattes, lioutikov}@kit.edu · 2026
Read SIR: Structured Image Representations for Explainable Robot Learning by Paul Mattes, Jan Schwab, Jens Bosch, Nils Blank, Maximilian Xiling Li, Minh-Trung Tang, Moritz Haberland, Rudolf Lioutikov, [0.5em] Intuitive Robots Lab, Karlsruhe Institute of Technology, Germany Robotics Institute Germany[0.3em] {paul.mattes, lioutikov}@kit.edu (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-30101.txt.
Description
Paul Mattes and seven colleagues make robot policies easier to understand by giving them structured eyes. Learned visual embeddings drive most robot policies today. Those embeddings are opaque and easily distracted by irrelevant detail. The team's method, SIR, routes vision through a scene graph instead. Scene graphs list objects and their relations explicitly.
SIR starts with a fully connected graph built from image features. A learned module then sparsifies that graph end to end. Only a task-relevant sub-graph reaches the action generation model. This design makes the policy intrinsically explainable by construction. On the RoboCasa benchmark, sparse graph policies beat image-based baselines. Average success rises clearly above the image-based comparison systems. The learned graphs also serve as a diagnostic tool. Deviations from human expectation expose dataset biases, including spurious correlations. Positional biases in the training data surface the same way.
Catalog record
| CREATOR | Paul Mattes, Jan Schwab, Jens Bosch, Nils Blank, Maximilian Xiling Li, Minh-Trung Tang, Moritz Haberland, Rudolf Lioutikov, [0.5em] Intuitive Robots Lab, Karlsruhe Institute of Technology, Germany Robotics Institute Germany[0.3em] {paul.mattes, lioutikov}@kit.edu |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 74 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2606.30101