Linguistic properties and model scale in brain encoding: from small to compressed language models

Subba Reddy Oota, Vijay Rowtula, Satya Sai Srinath Namburi, Khushbu Pahwa, Anant Khandelwal, Manish Gupta, Tanmoy Chakraborty, Bapi S. Raju · 2026

Read Linguistic properties and model scale in brain encoding: from small to compressed language models by Subba Reddy Oota, Vijay Rowtula, Satya Sai Srinath Namburi, Khushbu Pahwa, Anant Khandelwal, Manish Gupta, Tanmoy Chakraborty, Bapi S. Raju (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-07547.txt.

Description

Bigger language models align better with brain activity, but is scale itself the driver, or could smaller models suffice? Subba Reddy Oota and seven co-authors compare full-precision models, small models, and compressed quantized and pruned variants by predicting fMRI responses during natural language comprehension.

Across families up to 14 billion settings, 3-billion-size small models match larger models in brain predictivity. 1-billion models degrade, and linguistic properties and scale interact in shaping match. The study asks what minimal capacity captures brain-relevant representations. Brain experts and model builders get evidence that brain match does not require the largest models, just enough of the right capacity.

Catalog record

CREATORSubba Reddy Oota, Vijay Rowtula, Satya Sai Srinath Namburi, Khushbu Pahwa, Anant Khandelwal, Manish Gupta, Tanmoy Chakraborty, Bapi S. Raju
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE89 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602-07547

Browse the full library · About the Tower of Babel