How Much is Brain Data Worth for Machine Learning?
Lane Lewis [email protected], [0.5em] Zhixin Wang [email protected], [0.5em] David Schwab [email protected], [0.5em] Xaq Pitkow [email protected], [1em] Neuroscience Institute, Carnegie Mellon University, Pittsburgh, PA, USA · 2026
Read How Much is Brain Data Worth for Machine Learning? by Lane Lewis [email protected], [0.5em] Zhixin Wang [email protected], [0.5em] David Schwab [email protected], [0.5em] Xaq Pitkow [email protected], [1em] Neuroscience Institute, Carnegie Mellon University, Pittsburgh, PA, USA (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-09243.txt.
Introduction
Lewis and three colleagues ask how much brain data is worth for machine learning. If a person can solve a task, do brain measurements ease training a model for that task? Recent NeuroAI work suggests adding neural recordings to task training can improve performance and robustness, but when to expect a benefit, or how large, is unclear.
They formulate the question mathematically with a simple linear Gaussian model of task goals and recordings. For an estimator trained on brain data and task labels, they derive scaling laws linking performance to counts. From these laws come value and exchange rates between brain and task samples. Those rates quantify what extra task samples neural data is worth, given alignment, noise, dimension, and sample size. They also analyse test distribution shift, where brain-regularised learning gains robustness through invariances. Under a fixed budget, they characterise when brain data is worth collecting.
Catalog record
| CREATOR | Lane Lewis [email protected], [0.5em] Zhixin Wang [email protected], [0.5em] David Schwab [email protected], [0.5em] Xaq Pitkow [email protected], [1em] Neuroscience Institute, Carnegie Mellon University, Pittsburgh, PA, USA |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 148 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.09243
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
How Much is Brain Data Worth for Machine Learning? Lane Lewis [email protected], [0.5em] Zhixin Wang [email protected], [0.5em] David Schwab [email protected], [0.5em] Xaq Pitkow [email protected], [1em] Neuroscience Institute, Carnegie Mellon University, Pittsburgh, PA, USA Abstract If a person can solve a task, can measuring their brain make it easier to train a model to solve that task too? Recent NeuroAI work suggests that supplementing task training with neural recordings can modestly improve model performance and robustness. However, it is unclear when there should be a benefit from using neural data and how much benefit to expect. We formulate this question mathematically, and begin to address it theoretically using a simple, analytically tractable linear gaussian model of task targets and neural recordings. For a multimodal estimator trained on both brain data and task labels, we derive scaling laws for how performance scales with the numbers of brain and task samples. From these laws we derive relative value and exchange rates between brain samples and task samples, quantifying how much extra task samples neural data is worth as a function of task-brain alignment, neural and task noise, latent dimension, and brain data sample size. We also analyze test distribution shift, to identify conditions where brain-regularized learning can produce substantial robustness gains through learned invariances. Finally, under a fixed collection budget, we characterize the regimes in which brain data is worth collecting. Our results provide a foundation for understanding how valuable brain data could be for improving machine learning. 1 Introduction Modern machine learning (ML) systems often improve predictably as training resources scale [ 17 ] . In many settings, test performance depends systematically on factors such as dataset size, model capacity, and compute, giving rise to empirical and theoretical scaling laws. Understanding these laws is important both scientifically and practically: they help identify which resources are limiting performance and which interventions can most effectively improve sample efficiency and generalization. A natural question is whether brain data can act as another useful training resource. Humans and animals solve many tasks that overlap with those studied in machine learning, and neural recordings provide a partial view into the internal representations supporting this behavior. This suggests a form of brain distillation , in which a learner has access not only to input-output task data, but also to neural measurements from an expert biological system. Recent NeuroAI work has explored this idea by regularizing machine learning models with neural recordings, encoding models, or other brain-derived signals, with several studies reporting modest gains in task performance or robustness [ 6 , 20 , 7 ] . Despite this promise, it remains unclear when brain data should help at all, and how much improvement should be expected. Existing empirical results are often small in magnitude and difficult to interpret: gains may depend strongly on data regime, recording quality, task difficulty, or the alignment between recorded neural features and the task of interest. In some cases, apparent benefits may arise from relatively simple regularization effects rather than from genuinely useful task relevant structure in neural data [ 21 ] . As a result, current empirical work offers limited guidance on basic questions such as: when does brain data improve sample efficiency over task-only learning? How should the value of brain data scale with the number of task labels? What properties of the recordings determine whether brain data is useful? And when is collecting brain data worth its high cost? In this paper, we study these questions theoretically through a linear gaussian model of task targets and neural recordings. We analyze a multimodal estimator that uses both brain data and task targets, and compare it to task-only learning. Within this model, we derive explicit test error scaling laws in the numbers of brain and task samples. These scaling laws show how brain data can improve task sample efficiency, and how this improvement depends on quantities such as task-brain alignment, neural and task noise levels, neural latent dimension, and the amount of available brain data. We further derive an exchange rate between brain samples and task samples, which quantifies how much task supervision a given amount of neural data is worth. We also analyze test distribution shift, where our brain regularized estimator yields robustness gains by inducing useful invariances, and study a fixed-budget setting to characterize the regime when collecting new brain data makes sense. Our goal is not to provide a fully realistic model of biological representations or recordings. Rather, we aim to develop a tractable theoretical framework that isolates the main factors governing the value of brain data for machine learning. By making these tradeoffs explicit, our results provide a foundation for understanding when brain data should improve learning, how large those gains will be, and when additional recordings are worth collecting. 2 Related Work Brain Distillation. Brain-inspired machine learning dates back to the earliest stages of the field [ 27 ] . Existing approaches span a range of strategies, including biologically inspired architectures and learning rules [ 14 , 9 , 19 ] , connectomics-based approaches [ 30 ] as well as choosing models/data based on brain predictiveness [ 31 , 18 , 36 ] . Our work is most closely related to a more recent NeuroAI direction that uses neural recordings directly during training to guide machine learning models [ 6 , 20 , 25 , 32 ] . This paradigm has the practical advantage of being compatible with standard ML training pipelines and does not require a detailed mechanistic understanding of the underlying neural system. Within this line of work, several approaches have been explored, including fine-tuning pretrained models on brain data [ 32 , 8 , 25 , 6 ] , regularizing task models using neural encoding models [ 20 ] , and using neural data to guide decision boundaries [ 7 ] . Empirical studies have reported modest gains in task performance and robustness in some settings [ 6 , 21 ] . However, these gains are often difficult to interpret, since they may reflect generic regularization effects (such as noise [ 6 ] or low pass filtering [ 21 ] ) rather than genuinely task-relevant information extracted from neural recordings. Recent work further suggests that the value of brain data may be concentrated in low or hard to collect task sample regimes [ 23 ] . Despite this empirical literature, there is limited theoretical understanding of when brain data helps, by how much, and which neural signal properties determine its value. Our work addresses this gap by providing explicit scaling law analyses for a type of brain regularized estimator. Scaling Laws. Scaling laws have played a central role in modern machine learning, especially in language modeling, where they have been used to derive optimal training prescriptions under limited resources [ 13 , 17 ] . Related ideas have also begun to appear in neuroscience, including scaling analyses for brain decoding [ 34 , 3 , 4 ] and language encoding models in fMRI [ 2 ] . Multimodal scaling work further studies how performance depends jointly on multiple data sources [ 1 ] . Our work is distinct in that it studies multimodal scaling over brain data and task data , and derives an