Brain-Like Processing Pathways Form in Models With Heterogeneous Experts
Jack Cook, Danyal Akarca, Rui Ponte Costa, Jascha Achterberg · 2025
Read Brain-Like Processing Pathways Form in Models With Heterogeneous Experts by Jack Cook, Danyal Akarca, Rui Ponte Costa, Jascha Achterberg (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-02813.txt.
Description
The brain solves tasks by routing work through shifting coalitions of regions, but nobody knows the rules that build those coalitions. Jack Cook, Danyal Akarca, Rui Ponte Costa, and Jascha Achterberg probe the question in artificial networks built as mixtures of expert modules. Each expert differs from the others, like brain regions do.
Left alone, the experts never organized into pathways, which suggests the brain must add extra pressure. Three added biases did the trick: charging a cost for using fancy experts, easing that cost when the task goes badly, and randomly dropping experts during training. The resulting pathways mirrored how real brains mix cortex and deeper structures by task difficulty. The authors offer the setup as a lab for studying brain routing.
Catalog record
| CREATOR | Jack Cook, Danyal Akarca, Rui Ponte Costa, Jascha Achterberg |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 62 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2506-02813