More with Less: Bethel Allocation and Precision-Preserving Sample Size Reduction via Hierarchical Bayes Modelling

Siu-Ming Tam · 2026

Read More with Less: Bethel Allocation and Precision-Preserving Sample Size Reduction via Hierarchical Bayes Modelling by Siu-Ming Tam (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-17663.txt.

Introduction

Tam offers survey offices a way to shrink samples without losing precision. Demand for fine regional estimates grows while budgets stay flat or fall. His two-stage plan starts with Bethel allocation, which finds the smallest sample that meets all targets at once. A hierarchical Bayes small-area model then borrows strength across strata to cut further. Tests on a synthetic labour-force population of one million checked accuracy and coverage, he reports.

Catalog record

CREATORSiu-Ming Tam
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE45090
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.17663

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

More with Less: Bethel Allocation and Precision-Preserving Sample Size Reduction via Hierarchical Bayes Modelling Siu-Ming Tam Abstract Statistical offices face a familiar and intensifying dilemma: rising demand for detailed regional and domain-level estimates under budgets that are fixed or shrinking. National statistical offices (NSOs) either ignore the problem of optimal sample allocation for multiple target variables when designing a multi-purpose survey, or address it incorrectly — relying on ad hoc approaches such as computing Neyman allocations separately per variable and taking the element-wise maximum, a practice that simultaneously wastes budget and fails to guarantee precision across all domains. This paper presents a practical two-stage strategy that reframes the question: not how to allocate a given sample, but how small the sample can be made while still meeting pre-defined precision targets for all target variables across all geographic domains at once. The innovation lies not in inventing new methods, but in the novel combination of two well-established techniques applied to this cost-reduction problem: (i) multivariate constrained optimisation via Bethel allocation, which finds the globally minimum sample satisfying all precision constraints simultaneously; and (ii) Hierarchical Bayes (HB) small area modelling, which borrows strength across strata and permits a further reduction of the Bethel sample. The approach is validated using a Monte Carlo study ( B = 1,000 B=1{,}000 replications) based on a synthetic labour-force population of one million individuals, where known population truth allows rigorous evaluation of precision, accuracy, and credible-interval coverage. Keywords: Bethel allocation; Hierarchical Bayes; small area estimation; sample size reduction; multivariate optimisation; labour force survey; coefficient of variation. 1 Introduction Statistical agencies worldwide face a familiar and intensifying dilemma: rising demand for granular, timely, and domain-level statistics under budgets that are fixed or shrinking. The current threats to the traditional survey paradigm include declining response rates and rising operational costs, alongside growing expectations for more timely and disaggregated information ( National Academies 2017 ) . NSOs now operate in a rapidly shifting landscape marked by an explosion of new data sources, growing privacy concerns, declining survey response rates, and the rise of artificial intelligence, all of which demand a fundamental rethinking of how statistics are produced ( UNECE 2025 ) . In response, NSOs have explored a range of cost-reducing innovations. One prominent strand integrates big data and administrative data with traditional survey data to supplement or partially replace expensive data collection operations. Kim and Tam 2021 formalised data integration as a general tool for combining big data with probability sample data for finite population inference, demonstrating that hybrid estimators can be substantially more efficient than survey data alone. Tam 2024 extended this approach to small area estimation of binary variables, showing that a calibrated nearest-neighbour algorithm combining big data with survey data yields small area estimates that outperform standard Fay–Herriot and logit-normal HB methods when used without big data. Tam et al. 2023 showed that non-response follow-up resources — a major cost driver in probability surveys — can be allocated optimally to minimise mean squared error rather than simply to maximise response rates, yielding cost savings without sacrificing data quality. More broadly, UNECE 2022 documented that machine learning holds great potential for making statistical production more efficient by automating processes and enabling the use of new data sources. These strategies, however important, largely operate on the data after the sample has been designed. A complementary and under-explored approach is to ask, at the design stage itself: how small can the sample be made while still meeting all pre-specified precision requirements? This paper addresses precisely that question for the practically important case of a multi-purpose survey with multiple target variables and multiple geographic domains. The traditional approach to sample size determination in this setting relies on the Neyman allocation formula applied separately to each target variable, followed by taking the element-wise maximum of the resulting stratum sample sizes across variables. We show that this approach is doubly flawed: it over-samples for variables whose Neyman allocation is dominated, while simultaneously failing to guarantee domain-level precision for the variable that drives the maximum. The Bethel algorithm ( Bethel 1989 ) provides the correct solution, finding the smallest total sample that simultaneously satisfies the coefficient-of-variation (CV) target for every variable in every domain. Yet the Bethel sample size can itself be large, because it must protect precision using direct survey estimates alone. The key insight of this paper is that Hierarchical Bayes small area models ( Rao and Molina 2015 ) can be used to reduce the Bethel sample further. By borrowing strength across domains, HB models produce more precise domain estimates than direct estimates at the same sample size. This precision surplus can be exchanged for a smaller sample — one that, when combined with HB estimation, still meets all pre-defined precision targets. The result is a two-stage strategy: Bethel allocation provides the globally optimal design-based benchmark; HB modelling provides a principled means of going further. The approach is validated through a Monte Carlo simulation study ( B=1{,}000 replications) based on a synthetic labour-force population of one million individuals. The three target variables — employment status, unemployment status, and hours worked — span both binary and continuous types, enabling the illustration of both logit-normal binomial and Gaussian Fay–Herriot HB models. Known population truth at all levels permits rigorous evaluation of precision, accuracy, and credible-interval coverage under repeated sampling. The remainder of the paper is organised as follows. Section 2 establishes the design setting and notation. Section 3 presents Stage 1, explaining why the Neyman-max approach fails and how Bethel allocation corrects it. Section 4 presents Stage 2, describing the HB models, the prior calibration strategy, and the sample reduction algorithm. Section 5 discusses the shift in inference paradigm from design-based to model-based. Section 6 describes the simulation study design and evaluation criteria. Section 7 reports the numerical results. Section 8 identifies directions for further work, and Section 9 concludes. A complete, self-contained specification of the simulation data-generating process — including population structure, latent variable models, auxiliary covariate generation, and a full parameter summary — is provided in Appendix A . 2 Design Setting and Notation Let U=\{1,\ldots,N\} denote a finite population of N units partitioned into H non-overlapping strata, indexed h=1,\ldots,H , with stratum sizes N_{1},\ldots,N_{H} . A stratified simple random sample of size n_{h} is drawn without replacement from stratum h , giving a total sample of n=\sum_{h=1}^{H}n_{h} . Let c_{h}>0 denote the unit cost in stratum h and f_{h}=n_{h}/N_{h} the sampling fraction. The population is also partitioned into D+1 publication domains, indexed d=0,1,\ldots,D , where d=0 denotes the national domain and d=1,\ldots,D denote geographic sub-national domains. There are K target variables, indexed k=1,\ldots,K . For each variable–domain pair (d,k) , let Y_{d,k} denote the anticipated domain total and g_{d,k} the pre-specified upper bound on \mathrm{CV}(\hat{Y}_{d,k}) . Let S^{2}_{h,d,k} denote the within-stratum variance component for (d,k) , and \mathrm{DEFF}_{h,d,k} the corresponding design effect. The precision constraint for domain d and variable k is: \mathrm{CV}\!\left(\hat{Y}_{d,k}\right)\;\leq\;g_{d,k},\qquad\forall\;d=0,1,\ldots,D,\quad k=1,\ldots,K. (1) Throughout this paper, precision targets are set at g_{0,k}=0.03 (national CV \leq 3\% ) and g_{d,k}=0.08 (domain CV \leq 8\% for all d\geq 1 ) for all k . In the Australian Bureau of Statistics (ABS) terminology, CVs are referred to as Relative Standard

Read the complete text