StructLens: A Structural Lens for Language Models via Maximum Spanning Trees
Haruki Sakajo, Frederikus Hudi, Yusuke Sakai, Hidetaka Kamigaito, Taro Watanabe · 2026
Read StructLens: A Structural Lens for Language Models via Maximum Spanning Trees by Haruki Sakajo, Frederikus Hudi, Yusuke Sakai, Hidetaka Kamigaito, Taro Watanabe (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-03328.txt.
Description
Haruki Sakajo and four colleagues look inside language models through the structure of their layers. Interpretation work usually studies local links between tokens inside one layer, leaving the global pattern across layers unmapped. Their 2026 framework, StructLens, builds a maximum spanning tree over each layer's token representations, much as parsers build dependency trees for sentences, and compares layers by tree shape.
That structure-aware comparison paints a picture of layer similarity strikingly different from the standard cosine measure. Layers that look alike by content can differ in structure, and the reverse also holds. The signal proves practical as well: it guides layer pruning, the removal of layers to slim a model, better than content-based cues. The team releases its code for other researchers.
Catalog record
| CREATOR | Haruki Sakajo, Frederikus Hudi, Yusuke Sakai, Hidetaka Kamigaito, Taro Watanabe |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 83354 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.03328