StructLens: A Structural Lens for Language Models via Maximum Spanning Trees

Haruki Sakajo, Frederikus Hudi, Yusuke Sakai, Hidetaka Kamigaito, Taro Watanabe · 2026

Read StructLens: A Structural Lens for Language Models via Maximum Spanning Trees by Haruki Sakajo, Frederikus Hudi, Yusuke Sakai, Hidetaka Kamigaito, Taro Watanabe (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-03328.txt.

Description

Haruki Sakajo and four colleagues look inside language models through the structure of their layers. Interpretation work usually studies local links between tokens inside one layer, leaving the global pattern across layers unmapped. Their 2026 framework, StructLens, builds a maximum spanning tree over each layer's token representations, much as parsers build dependency trees for sentences, and compares layers by tree shape.

That structure-aware comparison paints a picture of layer similarity strikingly different from the standard cosine measure. Layers that look alike by content can differ in structure, and the reverse also holds. The signal proves practical as well: it guides layer pruning, the removal of layers to slim a model, better than content-based cues. The team releases its code for other researchers.

Catalog record

CREATORHaruki Sakajo, Frederikus Hudi, Yusuke Sakai, Hidetaka Kamigaito, Taro Watanabe
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE83354
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.03328

Browse the full library · About the Tower of Babel