Linguistic properties and model scale in brain encoding: from small to compressed language models
Subba Reddy Oota, Vijay Rowtula, Satya Sai Srinath Namburi, Khushbu Pahwa, Anant Khandelwal, Manish Gupta, Tanmoy Chakraborty, Bapi S. Raju · 2026
Read Linguistic properties and model scale in brain encoding: from small to compressed language models by Subba Reddy Oota, Vijay Rowtula, Satya Sai Srinath Namburi, Khushbu Pahwa, Anant Khandelwal, Manish Gupta, Tanmoy Chakraborty, Bapi S. Raju (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-07547.txt.
Description
Bigger language models align better with brain activity, but is scale itself the driver, or could smaller models suffice? Subba Reddy Oota and seven co-authors compare full-precision models, small models, and compressed quantized and pruned variants by predicting fMRI responses during natural language comprehension.
Across families up to 14 billion settings, 3-billion-size small models match larger models in brain predictivity. 1-billion models degrade, and linguistic properties and scale interact in shaping match. The study asks what minimal capacity captures brain-relevant representations. Brain experts and model builders get evidence that brain match does not require the largest models, just enough of the right capacity.
Catalog record
| CREATOR | Subba Reddy Oota, Vijay Rowtula, Satya Sai Srinath Namburi, Khushbu Pahwa, Anant Khandelwal, Manish Gupta, Tanmoy Chakraborty, Bapi S. Raju |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 89 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2602-07547