Dual system estimation using mixed effects loglinear models

Ceejay Hammond, Paul A. Smith, Peter G.M. van der Heijden · 2025

Read Dual system estimation using mixed effects loglinear models by Ceejay Hammond, Paul A. Smith, Peter G.M. van der Heijden (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-01359.txt.

Introduction

Official statisticians estimate a population's size by linking two lists and guessing how many people both missed. Ceejay Hammond, Paul Smith, and Peter van der Heijden study in 2025 how to do this across many regions at once. The standard approach fits separate numbers for each region, which multiplies parameters. Their alternative treats the regional effects as drawn from a shared distribution, described by just a mean and a spread.

Simulations show the shared model edges out the separate one on accuracy, if only slightly. The team also shows how the idea extends from two lists to several. The gain matters most when regions are many and each list is thin.

Catalog record

CREATORCeejay Hammond, Paul A. Smith, Peter G.M. van der Heijden
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE75 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-01359

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Dual system estimation using mixed effects loglinear models Ceejay Hammond, Paul A. Smith, Peter G.M. van der Heijden Abstract In official statistics, dual system estimation (DSE) is a well-known tool to estimate the size of a population. Two sources are linked, and the number of units that are missed by both sources is estimated. Often dual system estimation is carried out in each of the levels of a stratifying variable, such as region. DSE can be considered a loglinear independence model, and, with a stratifying variable, a loglinear conditional independence model. The standard approach is to estimate parameters for each level of the stratifying variable. Thus, when the number of levels of the stratifying variable is large, the number of parameters estimated is large as well. Mixed effects loglinear models, where sets of parameters involving the stratifying variable are replaced by a distribution parameterised by its mean and a variance, have also been proposed, and we investigate their properties through simulation. In our simulation studies the mixed effects loglinear model outperforms the fixed effects loglinear model although only to a small extent in terms of mean squared error. We show how mixed effects dual system estimation can be extended to multiple system estimation. Key Words Population size estimation; Capture-recapture; Simulation; Chapman estimator; Multiple system estimator 1 Introduction The dual system estimator is the most simple capture-recapture model and was originally used to estimate the size and density of wildlife populations (Seber, 1986). It is now commonly used across official statistics to estimate the population size for domains of interest. Two sources are linked and the unobserved number of units missed from both sources is estimated. In official statistics this estimator was originally used for coverage assessment in the 1950 decennial United States Census, to estimate coverage error (Wolter, 1986). This estimator relies on five key assumptions (International Working Group for Disease Monitoring and Forecasting, 1995) – that the population is closed, there is perfect matching between the two lists, the capture/inclusion probability of elements in at least one of the lists is homogeneous (Seber, 1982; van der Heijden et al ., 2012), the inclusion in the lists is independent and there are no erroneous captures in any list (no overcoverage). All of these key assumptions should be met, however, in practice it can be difficult to ensure this. One of the key assumptions that underpins dual system estimation is that the inclusion in the lists is independent. In practice independence can be violated for one of three reasons: (a) operationally, such as that the modes of data collection are the same for both lists, (b) conditional inclusion of units, where an individual’s inclusion in one list is conditional on their inclusion in the other list, and (c) failure of the homogeneous inclusion probability assumption (Brown, Sexton, Abbott and Smith, 2019). The simplest situation we consider for dual system estimation is by fitting a loglinear model with two dichotomous variables. To help to satisfy the homogeneous inclusion probability assumption, one can make use of one or more stratifying variables (Gerritse, Bakker and van der Heijden, 2015). The dual system estimation is then carried out at each of the levels of the stratifying variables. The methodology we discuss in this paper can be extended to loglinear models of any size, in terms of both the number of variables, and the number of levels of the stratifying variables. It suits our purposes to define loglinear models in terms of Poisson regression models with a log link (Nelder and Wedderburn, 1972). One can make use of fixed effects loglinear models, which explain within-group variability for each of the stratifying variables. This leads to models with many parameters. These models are inefficient for large numbers of groups with small numbers of observations within groups (Townsend, Buckley, Harada and Scott, 2013; Clarke, Crawford, Steele and Vignoles, 2015). Therefore we investigate the use of mixed effects loglinear models, also known as multilevel, hierarchical or variance components models (Hox, Moerbeek and Van de Schoot, 2017). These models replace the fixed parameters for the stratifying variables with normal distributions parameterised by their mean and variance (Berkhof and Snijders, 2001). Royle and Link (2002) present the use of random effects to model heterogeneous capture probabilities and of shrinkage estimation to account for the variability introduced by using sample data. Agresti (2013) points out that mixed effects loglinear models can be used to model heterogeneous capture probabilities across groups by borrowing strength from the whole. The multilevel model can also be used as a bias-robust approach, because it borrows information across all of the groups, and therefore suffers less from small sample bias; it provides an alternative to the Chapman estimator. Therefore, we expect these models to improve population estimates in terms of mean squared error, compared to the corresponding fixed effects loglinear models. We investigate the performance of mixed effects loglinear models through simulation, where population sizes are both small and large, and are made up of varying numbers of regions. For small sample sizes, bias can be a concern, as illustrated in recent contributions of work from Zult, van der Heijden and Bakker (2025); Rivest and Yauck (2024). In mixed effects loglinear model parlance, we are interested in estimating the number of individuals (level 1) within each of the regions (level 2). We also show how the methodology discussed for dual system estimation is extended to multiple system estimation, which makes use of more than two lists to estimate the population for domains of interest. The structure of the paper is as follows. Section 2 discusses the methodology for fixed effects loglinear models for the simplest situation involving tables without and with a stratifying variable. We discuss mixed effects loglinear models, including shrinkage estimation and model fitting. Section 3 describes the set-up of the simulation study; we examine random effects which are normally distributed in line with the normality assumption, and also lognormally or Pareto distributed to evaluate the effect of departures from the assumption. Section 4 presents the results, where fixed and mixed effects loglinear models are compared by the mean absolute relative bias and the coefficient of variation for the total population and for the levels of the stratifying variable. Section 5 shows how the methods are extended to multiple system estimation. Section 6 makes a comparison, including a discussion of advantages and disadvantages. 2 Methodology - Dual System Estimation 2.1 Fixed effects loglinear models 2.1.1 Models for two-way tables The simplest situation that we consider is the fixed effects loglinear model with two dichotomous variables A and B . Let A be indexed by i=0,1 and B by j=0,1 . We denote the expected count for cell (i,j) of the two-way contingency table of variables A and B by \mu_{ij} . We denote the loglinear parameters by \lambda . Then the saturated loglinear model for two lists is \log\mu_{ij}=\lambda+\lambda_{i}^{A}+\lambda_{j}^{B}+\lambda_{ij}^{AB}, (1) where \lambda is the intercept term, \lambda_{i}^{A} and \lambda_{j}^{B} are the respective parameters for variables A and B, and \lambda_{ij}^{AB} is the two factor interaction parameter. The parameters are identified by setting them equal to 0 when an index is 0, i.e. \lambda_{0}^{A}=\lambda_{0}^{B}=\lambda_{00}^{AB}=\lambda_{01}^{AB}=\lambda_{10}^{AB}=0 , referred to as the corner point constraint parameterization. Thus the model has

Read the complete text