Exhaustive Circuit Mapping of a Single-Cell Foundation Model Reveals Massive Redundancy, Heavy-Tailed Hub Architecture, and Layer-Dependent Differentiation Control
Ihor Kendiukhov · 2026
Read Exhaustive Circuit Mapping of a Single-Cell Foundation Model Reveals Massive Redundancy, Heavy-Tailed Hub Architecture, and Layer-Dependent Differentiation Control by Ihor Kendiukhov (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-11940.txt.
Description
Kendiukhov maps the circuits of Geneformer, a single-cell foundation model, by tracing every active feature at one layer. Selective sampling in past work risked bias, so he traces all 4,065 sparse autoencoder features and finds nearly 1.4 million strong downstream links. The wiring is heavy-tailed: 1.8% of features act as hubs, and 40% of the top hubs lack any biological label. Ablating feature triplets shows redundancy grows with interaction order, with no synergy. Steering tests tie late-layer features to pushing cells toward maturity.
Catalog record
| CREATOR | Ihor Kendiukhov |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 31332 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.11940