Exhaustive Circuit Mapping of a Single-Cell Foundation Model Reveals Massive Redundancy, Heavy-Tailed Hub Architecture, and Layer-Dependent Differentiation Control

Ihor Kendiukhov · 2026

Read Exhaustive Circuit Mapping of a Single-Cell Foundation Model Reveals Massive Redundancy, Heavy-Tailed Hub Architecture, and Layer-Dependent Differentiation Control by Ihor Kendiukhov (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-11940.txt.

Description

Kendiukhov maps the circuits of Geneformer, a single-cell foundation model, by tracing every active feature at one layer. Selective sampling in past work risked bias, so he traces all 4,065 sparse autoencoder features and finds nearly 1.4 million strong downstream links. The wiring is heavy-tailed: 1.8% of features act as hubs, and 40% of the top hubs lack any biological label. Ablating feature triplets shows redundancy grows with interaction order, with no synergy. Steering tests tie late-layer features to pushing cells toward maturity.

Catalog record

CREATORIhor Kendiukhov
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE31332
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.11940

Browse the full library · About the Tower of Babel