Measurement Induced Confounding
\addORCIDlink George Perrett0000-0002-1930-2581 and \addORCIDlink Klint Kanopka0000-0003-3196-9538 · 2026
Read Measurement Induced Confounding by \addORCIDlink George Perrett0000-0002-1930-2581 and \addORCIDlink Klint Kanopka0000-0003-3196-9538 (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-28774.txt.
Introduction
Perrett and Kanopka identify a hidden flaw in studies that try to remove the influence of traits nobody can observe directly. Motivation, self-belief and ability often confound results, meaning they distort the link between a treatment and its outcome. Researchers usually adjust for such traits using test or survey answers, treating those answers as if they were exact.
Measurement error in those answers, the authors show, does not wash out. It spreads through the calculations and leaves the estimated average treatment effect biased, with uncertainty ranges that are also wrong. In other words, many published studies that adjusted for hidden traits may report effects that are off and confidence that is misplaced. Their remedy is a Bayesian joint estimation approach, which fits the treatment assignment model, the measurement model and the outcome model together in one step.
Catalog record
| CREATOR | \addORCIDlink George Perrett0000-0002-1930-2581 and \addORCIDlink Klint Kanopka0000-0003-3196-9538 |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 51 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2606.28774
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Measurement Induced Confounding \addORCIDlink George Perrett0000-0002-1930-2581 and \addORCIDlink Klint Kanopka0000-0003-3196-9538 Abstract A critical assumption of observational studies is that all confounding variables must be known and sufficiently adjusted for to estimate causal effects. An implicit, and often overlooked, aspect of this assumption is that all confounding variables have been measured without error. In the social and medical sciences, latent traits such as motivation, self-efficacy, and ability measures are likely confounding variables. Because latent traits are not directly observable, conventional approaches to adjust for them in observational studies rely on collecting responses to individual items on a test or survey instrument and then adjust for sum scores, measurement model-derived ability estimates, or item responses directly. Through a process we describe as measurement induced confounding , we show that measurement error propagates through the estimation process and that current conventional approaches to adjusting for latent traits in observational studies produce biased estimates of the average treatment effect with incorrectly calibrated coverage properties. A critical implication of this finding is that current observational studies that attempt to adjust for latent confounding variables likely put forth biased causal estimates with incorrect uncertainty intervals. We show that measurement induced confounding can be resolved through a Bayesian Joint Estimation approach that simultaneously estimates the measurement model, the treatment assignment model, and the response model. Measurement Induced Confounding \leftheader Perrett and Kanopka \authorsaffiliations New York University \authornote Correspondence concerning this article should be addressed to George Perrett, Department of Applied Statistics, Social Science, and Humanities, New York University, 246 Greene St., Floor 3, New York, NY, 10003. E-mail: [email protected] keywords Causal Inference, Observational Studies, Confounding, Latent Traits, Psychometrics, Item Response Theory 1 Introduction When randomized studies are ethically or logistically infeasible, researchers have no choice but to turn to observational studies to answer causal questions. The primary limitation of observational studies is the assumption that all confounding variables are known and appropriately conditioned on. Work on propensity score methods ( 2 ; 44 ; 10 ) and recent advances in machine learning for causal inference ( 24 ; 18 ; 22 ) have focused on sufficiently conditioning on measured confounding variables without making unnecessary parametric modeling assumptions. An overlooked subtlety of these approaches is that they not only assume all confounding variables are measured but that they have been measured without error . Methodological work in causal inference often begins from the position that there is zero measurement error in all confounding variables ( 12 ) . In practice, assuming all confounders are measured without error is problematic because confounding variables in the social, educational, and medical sciences commonly include measures of latent traits, including—but not limited to—mathematical ability ( 38 ) , well-being ( 54 ) , personality ( 55 ) , depression ( 51 ) , and motivation ( 33 ) . Methodological work in psychometrics specifically concerned with the measurement of these sorts of latent constructs explicitly assumes that latent traits are measured with error. To illustrate the problem posed by confounding variables that are latent traits, we will consider a hypothetical observational study on the causal effect of attending college on income. Knowing the causal effect of college education on income is a pressing policy question with great societal implications, yet running a randomized study to answer it is impossible, leaving us no choice but to rely on observational methods. 1 1 1 Randomizing students to either attend or not attend college is ethically non-permissible. For simplicity, suppose we knew that motivation was the only confounding variable and student motivation was linearly related to the probability that they attend college and also linearly related to their income. Here motivation meets the classical definition of a confounding variable and failing to sufficiently adjust for motivation would bias our estimate of the causal effect of college attendance on income. As such, in order to estimate the causal effect of college attendance on income, we must sufficiently adjust for motivation. The problem we face is that motivation is a latent trait that can not be directly observed. To measure motivation, we rely on instruments such as the achievement motivation scale ( 8 ) or Motivational Trait Questionnaire ( 19 ) . In our hypothetical observational study, we will imagine we have collected responses to a 100-item instrument designed to measure motivation in this population. Given our 100 item responses per subject, there are multiple strategies for attempting to adjust for motivation. A reasonable first approach is computing a sum score by adding each individual’s 100-item responses together. This method is commonly used in the social sciences but it does not consider the possibility that some items may reveal more about an individual’s motivation than others. Item Response Theory (IRT; 35 ) provides a framework that allows discrete items with categorical responses to provide continuous estimates of latent motivation. Adjusting for an IRT estimate of motivation offers greater flexibility than the sum-score approach, as it also allows items to contribute differential amounts of information. While more flexible IRT models and sum scoring differ in how they leverage information from item responses, they are similar in that they both invoke a model to collapse information from many items into a single score of the latent trait of interest. A recent branch of psychometrics work has advocated for directly using item information and avoid collapsing information into a single metric ( 6 ; 16 ) , and following these lines of research, rather than adjusting for a single summary of the item response string, we could also attempt to control for motivation by adjusting for the responses to all 100 items directly. In this work, we argue that adjusting for sum scores, IRT-derived scores, or even individual item responses themselves are all insufficient to condition on confounding from latent traits. Through a process we describe as measurement induced confounding , measurement error propagates through the adjustment process, resulting in biased estimates and undercoverage of causal effects. An uncomfortable implication of this work is that any observational study that uses conventional approaches to adjust for latent trait confounders may present both biased parameter estimates and has incorrect coverage properties. Fortunately, there is a path forward: We propose using Bayesian Joint Estimation of the measurement model, the response surface, and the assignment mechanism to address measurement-induced confounding. Before we can formalize the problem of measurement induced confounding and outline Bayesian Joint Estimation as a solution, we first begin by providing a brief overview of Item Response Theory and causal inference. 2 Measurement and Item Response Theory A common challenge in behavioral research and the social sciences is the measurement of latent traits , or qualities of an individual that are not themselves directly observable. Typically, the levels of these traits need to be inferred from something that is directly observable (e.g., responses to individual items on a test or survey, performances in structured tasks, etc.). We refer to any individual piece of observed information as an item and represent individual i ’s response to item j as X_{ij} , denoting an individual realization of that response as x_{ij} . In the case of dichotomous items that are scored as correct/incorrect (or endorsed/not endorsed), we have X_{ij}\in\{0,1\} , with X_{ij}=1 when individual