The purpose of an estimator is what it does: Misspecification, estimands, and over-identification

Isaiah Andrews, Jiafeng Chen, and Otavio Tecchio · 2025

Read The purpose of an estimator is what it does: Misspecification, estimands, and over-identification by Isaiah Andrews, Jiafeng Chen, and Otavio Tecchio (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-13076.txt.

Introduction

Isaiah Andrews, Jiafeng Chen, and Otavio Tecchio rethink estimation in economics in twenty twenty-five. In over-identified models, the model is usually wrong in some way. Misspecification is the norm, not the exception. Then different estimators stop being rival routes to one target. Each estimator defines its own estimand, its own answer.

The authors review recent use of generalized method of moments in a top journal. Formal specification testing is rare there, they find. Estimators that would be inefficient under a correct model are widely used, including hand-picked moments. They synthesize recent theory into guidelines for transparent, robust work. A new result shows Hansen's J-statistic measures the range of estimates reachable at a given precision. Readers get a clearer contract between method and meaning.

Catalog record

CREATORIsaiah Andrews, Jiafeng Chen, and Otavio Tecchio
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE96 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2508-13076

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

The purpose of an estimator is what it does: Misspecification, estimands, and over-identification Isaiah Andrews, Jiafeng Chen, and Otavio Tecchio Abstract . In over-identified models, misspecification—the norm rather than ex- ception—fundamentally changes what estimators estimate. Different estimators imply different estimands rather than different efficiency for the same target. A review of recent applications of generalized method of moments in the American Economic Review suggests widespread acceptance of this fact: There is little formal specification testing and widespread use of estimators that would be inefficient were the model correct, including the use of “hand-selected” moments and weighting matrices. Motivated by these observations, we review and synthesize recent results on estimation under model misspecification, providing guidelines for transparent and robust empirical research. We also provide a new theoretical result, showing that Hansen’s J-statistic measures, asymptotically, the range of estimates achievable at a given standard error. Given the widespread use of inefficient estimators and the resulting researcher degrees of freedom, we thus particularly recommend the broader reporting of J-statistics. 1. Introduction Scientific and policy questions in economics frequently hinge on quantities such as counterfactual outcomes or effects on welfare. Recovering them often requires strong modeling assumptions. At the same time, many economists concede that economic models are at best approximately true ( White 1982 ; Hansen and Sargent 2008 ; Manski 2013 ; Rodrik 2015 ) . This leads to a gap between formal econometric recommendations and empirical practice for over-identified models. 1 1 1 That is, models which admit different estimators for the same parameter and consequently imply testable restrictions on the data distribution, c.f. Chen and Santos 2018 . Many textbook treatments of over-identified models ( Hayashi 2000 ; Stock and Watson 2018 ; Davidson and Mackinnon 2021 , e.g.) recommend researchers test whether the model’s over-identifying restrictions hold and use an efficient estimator when failing to reject. Experience reading economics papers and attending economics seminars, however, suggests that empirical practice often flouts these prescriptions. To document this, we examined empirical papers published in the American Economic Review between 2020 and 2024 which fit models using the generalized method of moments ( Hansen 1982 , GMM,) or related methods, obtaining a sample of 36 papers, of which 22 are over-identified. In this sample, we find that researchers commonly fit models using only a subset of the model’s implications, for instance fitting a fully parametric model by moment-matching rather than maximum likelihood. Researchers also commonly weight moments differently than would the optimal estimator (e.g. 14 of the over-identified papers in our sample report no efficiently-weighted specifications). Both practices yield inefficient estimators under correct model specification. Formal tests of over-identifying restrictions are notably scarce (only 3 papers in our sample report J-tests of overidentifying restrictions), though various “eyeball tests” are common, with researchers e.g. plotting the gap between data- and model-implied moments (25 papers in our sample). This gap between econometrics as taught and econometrics as practiced could appear puzzling if one thought economic models were plausibly correct—after all, it would be better to use a correct model, and to use the best estimator under that model, than to persist in using an incorrect model or an inefficient estimator. The common practices outlined above are, on the other hand, reasonable under the premise that any model we can take to the data is almost certainly wrong. Consistent with this premise, many papers in our data show evidence of substantial misspecification, with 8 papers reporting just-identified specifications where the parameter estimates cannot set the moments to zero. Under this view, it is natural to fit the model aiming for the “least-bad” answer. In judging model fit, it is moreover natural to draw quantitative judgments about where and how well the model fits, rather than focusing on the all-or-nothing question of whether the model fits exactly. A similar perspective underlies a large and active recent literature in econometrics which studies the behavior and interpretation of popular estimators without assuming that the model is correct. This article provides a selective synthesis of this literature. While one can reasonably critique some of this literature as a reverse-engineering exercise, starting from estimators and working backwards in the hope of finding interpretable estimands ( Mogstad and Torgovitsky 2024 ) , the perspective of this literature can also reasonably be summarized as ‘‘the purpose of an estimator is what it does.’’ 2 2 2 Paraphrased from Beer 1985 —“The purpose of a system is what it does. There is, after all, no point in claiming that the purpose of a system is to do what is consistently fails to do.” That is, given that economic models are generally misspecified, it is important to understand how estimators actually behave in applications. Moreover, in our view, it can be reasonable to choose among estimators on this basis. While reasonable, the common practice of choosing estimators based on misspecification concerns increases researcher degrees of freedom ( Simmons et al. 2011 ) . Different researchers estimating the same economic model on the same data may reach different conclusions due to differing misspecification concerns and subsequent estimation choices. Worse, dishonest researchers could use misspecification concern as a pretext for estimation choices which deliver a desired result, e.g. rejection of some null hypothesis. As a first step towards quantifying the scope for such manipulation, and motivated by the wide range of weight-selection approaches observed in our survey of AER papers, we consider the choice of weight matrices for a given (over-identifying) set of moments. In models where the degree of misspecification is on the same order as sampling uncertainty (i.e. locally misspecified models) we prove that, asymptotically, the set of estimates obtainable at a given standard error is an interval centered at the efficient GMM estimate, with width proportional to the square root of Hansen 1982 ’s J-statistic. Moreover, we show that a nefarious researcher can choose weights to engineer a t-statistic value of at least \sqrt{J} for any null hypothesis. Thus, given a set of moments, J-statistics measure the scope for a nefarious researcher to “weight hack” their findings. To summarize our conclusions, we recommend the following: (1) Researchers should distinguish concerns and diagnostics for correct econometric specification (i.e. does the model correctly describe the relationship between the observable data and the objects of economic interest) vs. correct statistical specification (i.e. does the model fit the data distribution). (2) Researchers should be explicit about what their choice of estimator implies for their estimand. If they use estimators which are inefficient under the model (e.g. due to non-optimal weighting), they should articulate why they believe such estimators are preferable. (3) Researchers should use misspecification-robust standard errors, which excludes the usual GMM standard errors ( Hansen 1982 ) in over-identified models. Formulas are available for many problems, and when in doubt the bootstrap will often work. (4) Researchers estimating over-identified moment condition models should report J-statistics, since they summarize the range of results may be obtained via alternative weighting choices. J-statistics are thus important to report even if one dispenses with J-tests. The following sections develop these points in detail, highlighting relevant references from the literature as we proceed. Section 2 defines what we mean by econometric and statistical models, illustrating with examples, and introduces corresponding notions

Read the complete text