Feature leakage and the identifiability of direct-dependency entropy models of neural activity
Houman Safaai, Bernardo L. Sabatini, [0.5em] Kempner Institute for the Study of Natural and Artificial Intelligence at Harvard University · 2026
Read Feature leakage and the identifiability of direct-dependency entropy models of neural activity by Houman Safaai, Bernardo L. Sabatini, [0.5em] Kempner Institute for the Study of Natural and Artificial Intelligence at Harvard University (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-01661.txt.
Introduction
Safaai and Sabatini ask what simple neural models really prove about computation. Neurons receive thousands of inputs on branching dendrites. Population activity is often modeled with direct input-output rules. Each input adds independently to a scalar drive in such rules. Good prediction by those models may not reveal the mechanism. For maximum-entropy models matched to rates and coactivities, explained entropy is a prediction measure. It is not a mechanism-identification test, they argue. Omitted terms can be absorbed into first-order parameters when correlated.
They introduce diagnostics that separate prediction from rule recovery. State reweighting holds the response rule fixed while input spreads change. Log-odds contrasts and temporal leakage controls join the toolkit. In simulations, higher-order responses pass naive tests but get caught after reweighting. CA1 recordings show many tables turn distribution-sensitive under reweighting.
Catalog record
| CREATOR | Houman Safaai, Bernardo L. Sabatini, [0.5em] Kempner Institute for the Study of Natural and Artificial Intelligence at Harvard University |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 75 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2606.01661
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Feature leakage and the identifiability of direct-dependency entropy models of neural activity Houman Safaai, Bernardo L. Sabatini, [0.5em] Kempner Institute for the Study of Natural and Artificial Intelligence at Harvard University Abstract Biological neurons receive thousands of synaptic inputs on branching, electrically excitable dendrites. Many local nonlinear interactions between coactive synapses have been described, yet population activity is often modeled with simpler direct input-output rules in which each input contributes independently to a scalar drive. A natural question is therefore not only whether such direct models predict neural activity, but what their success reveals about the computation carried out by the neuron. We study this question for conditional maximum-entropy (MaxEnt) models that match the output rate and pairwise output-input coactivities. These models provide a principled measure of prediction by first-order sufficient statistics. However, as we describe, the associated entropy calculation cannot be used as a mechanism-identification test. A restricted MaxEnt fit is an information projection under the sampled input distribution. Omitted interaction, temporal, or hidden-state terms can be absorbed into fitted first-order parameters whenever they are correlated with the included statistics, yielding models with high goodness of fit that are mechanistically incorrect. For sparse correlated binary inputs, this absorption has a simple coskewness form that we derive explicitly. We introduce diagnostics that separate in-distribution prediction from recovery of the response rule. These include (1) a state-reweighting control that holds P ( y ∣ x ) P(y\mid x) fixed while changing P ( x ) P(x) (using a mild product-marginal control and a balanced stress-test endpoint); (2) conditional log-odds contrasts that test additivity of the local response table; and (3) a temporal leakage simulation that quantifies when instantaneous statistics absorb lagged mechanisms. In ground-truth simulations, purely higher-order responses can pass first-order entropy and raw coactivity tests under leakage-prone sampling, whereas the same response is correctly classified after reweighting. We apply the diagnostics to selected, leakage-enriched local tables from CA1 hippocampal recordings. We examine diagnostic subsets chosen to maximize first-order predictability, not as a representative survey of CA1 activity. In these tables, approximately half of those that appear first-order under empirical weights become distribution-sensitive under balanced reweighting, an effect far above a matched additive-surrogate null. An unselected control, analyzed conditional on empirical-success tables, shows that removing the output-based input selection does not remove the effect. This shows how large the absorption can be under the sampling geometry of the data to which these models are applied. In short, these models do not provide an estimate of the fraction of neural responses that are mechanistically higher-order. Direct entropy-explained fractions and raw coactivity predictions are therefore best interpreted as predictions under the observed state distribution, not as evidence that mechanisms outside the direct model are absent or small. 1 Introduction Biological neurons integrate synaptic inputs on complex dendritic trees, where voltage-dependent conductances, local synaptic interactions, plateau potentials, and spatially-confined integration can make the response to coactive inputs nonlinear [ 19 , 20 , 21 , 22 ] . At the same time, much of theoretical neuroscience and machine learning uses a simpler description of neural computation: each input contributes independently to a summed drive, and the neuron applies a scalar nonlinearity to produce an output. How much of the activity of an individual neuron results from interactions between its inputs is unknown. Answering this question will require obtaining large-scale simultaneous recordings of the activity patterns of an individual neuron and all of its inputs. The need to model such data motivates the statistical and computational question: if an additive input-output model (i.e. a linear model with no interactions between inputs) predicts neuronal activity well, what, if anything, has been learned about the underlying mechanism that generated the neural response? Conditional maximum entropy offers a potential method to address the predictive part of this question. In these MaxEnt approaches, one models a binary output by the least-structured conditional distribution that matches the output rate and the pairwise output-input coactivities, and examines how much activity remains unexplained. A recent application of MaxEnt to hippocampal, visual cortex, and C. elegans neural activity reported that such direct models predict much of the raw higher-order and time-delayed coactivities in the datasets [ 1 ] . Those results are important as statements about prediction. The harder question is whether they identify the absence, or relatively small contributions, of mechanisms outside the direct model. The gap between prediction and interpretation is familiar from the study of model misspecification [ 7 , 6 ] : a restricted model can fit data well for the wrong structural reason. The issue is sharpened by the empirical distribution of recorded input states. Entropy explained by a direct model, and prediction of raw coactivities, are averages over that distribution. If omitted interaction or temporal features are correlated with the direct sufficient statistics on the sampled states, their effects can be folded into the fitted first-order parameters. This problem is especially acute for sparse binarized recordings. Binarization collapses positive powers of a single input because x^{n}=x on \{0,1\} for all n\geq 1 , and correlations among sparse binary inputs create a coskewness channel through which interaction terms project onto first-order terms. Sparse high-dimensional data can also downweight or miss the input states that distinguish an additive response from an interaction response. As a result, the same conditional response can yield very different first-order attribution under different state weights, even though the response mechanism itself has not changed. Here, we develop an identifiability framework for direct-dependency conditional MaxEnt models of neural activity. We first use an information-projection calculation to show how omitted statistics can shift fitted first-order MaxEnt parameters and leave only residual structure detectable (Eq. 8 ). We then derive a coskewness identity for sparse binary inputs, which makes the projection mechanism explicit (Eq. 18 ). These calculations motivate three diagnostics: state reweighting, conditional log-odds contrasts, and a temporal leakage control. We apply the diagnostics to ground-truth simulations and to publicly available CA1 and visual-cortex recordings. The framework is not meant to dismiss first-order models, and it certainly does not imply that higher-order mechanisms dominate every dataset. It provides quantitative tools for stating the predictive contribution of a direct-dependency MaxEnt model with appropriate scope. In summary, prediction by first-order sufficient statistics under P_{\mathrm{emp}}(x) is a well-defined and useful descriptor, but it does not identify the absence of mechanisms outside the direct model. It cannot distinguish, on its own, whether biological neurons nonlinearly integrate synaptic information or are effectively additive logistic units under the sampled conditions. Related cautions about restricted maximum-entropy population models, coupled GLMs, higher-order correlations, dendritic nonlinearities, and activity-based connectivity inference appear in prior work [ 8 , 12 , 15 , 14 , 16 , 13 , 20 , 21 , 19 ] ; the framework here addresses identifiability in a unified way and applies to any direct-dependency MaxEnt analysis of binarized neural data. Figure 1: State weighting can determine whether a fixed response table appears first-order. Bubble color is the true response P(y=1\mid x) , bubble area is the state weight P(x) , shaded backgrounds are best first-order readouts and dashed lines mark the \hat{P}=0.5 boundary. a–c A pure XOR-like interaction is well described by a first-order readout under empirical rare-corner sampling