ICE-ID: A Novel Historical Census Data Benchmark Comparing NARS against LLMs, & a ML Ensemble on Longitudinal Identity Resolution
Gonçalo Hora de Carvalho, Lazar S. Popov, Sander Kaatee, Kristinn R. Thórisson, Tangrui Li, Pétur Húni Björnsson, Jilles S. Dibangoye · 2025
Read ICE-ID: A Novel Historical Census Data Benchmark Comparing NARS against LLMs, & a ML Ensemble on Longitudinal Identity Resolution by Gonçalo Hora de Carvalho, Lazar S. Popov, Sander Kaatee, Kristinn R. Thórisson, Tangrui Li, Pétur Húni Björnsson, Jilles S. Dibangoye (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-13792.txt.
Introduction
Gonçalo Hora de Carvalho and co-authors released a benchmark in 2025 for matching people across old census records. Their Icelandic set holds nearly a million records from sixteen censuses spanning 1703 to 1920. Names follow the patronymic custom, farms nest inside parishes, and spellings drift over decades, all of which trip standard matching tools.
About a quarter of the records carry expert-made identity tags, the ground truth for scoring. The paper maps time coverage, missing fields, name clashes, and how hard it is to gather match candidates. It also sets a test rule that mimics real use across time. The full set, splits, rebuild scripts, and a browsing dashboard are shared. A companion paper reports how baseline models score on the task.
Catalog record
| CREATOR | Gonçalo Hora de Carvalho, Lazar S. Popov, Sander Kaatee, Kristinn R. Thórisson, Tangrui Li, Pétur Húni Björnsson, Jilles S. Dibangoye |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 46 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2506-13792
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
ICE-ID: A Novel Historical Census Data Benchmark Comparing NARS against LLMs, & a ML Ensemble on Longitudinal Identity Resolution Gonçalo Hora de Carvalho, Lazar S. Popov, Sander Kaatee, Kristinn R. Thórisson, Tangrui Li, Pétur Húni Björnsson, Jilles S. Dibangoye Abstract We introduce ICE-ID, a novel benchmark dataset for historical identity resolution, comprising 220 years (1703–1920) of Icelandic census records. ICE-ID spans multiple generations of longitudinal data, capturing name variations, demographic changes, and rich genealogical links. To the best of our knowledge, this is the first large-scale, open tabular dataset specifically designed to study long-term person-entity matching in a real-world population. We define identity resolution tasks (within and across census waves) with clearly documented metrics and splits. We evaluate a range of methods: handcrafted rule-based matchers, a ML ensemble as well as LLMs for structured data (e.g. transformer-based tabular networks) against a novel approach to tabular data called NARS (Non-Axiomatic Reasoning System) - a general-purpose AI framework designed to reason with limited knowledge and resources. Its core is Non-Axiomatic Logic (NAL), a term-based logic. Our experiments show that NARS is suprisingly simple and competitive with other standard approaches, achieving SOTA at our task. By releasing ICE-ID and our code, we enable reproducible benchmarking of identity resolution approaches in longitudinal settings and hope that ICE-ID opens new avenues for cross-disciplinary research in data linkage and historical analytics. 1 Introduction Linking historical census records is an important step in research on social mobility, demographic change, migration, and epidemiology, yet it remains arduous because names mutate, fields are missing, and administrative borders shift over time [ 18 ] . While supervised matchers trained on labelled pairs improved accuracy for nineteenth-century U.S. censuses [ 2 ] , most census-specific benchmarks still oversimplify the problem: they cover only short time ranges (often a single decade), omit kinship structure, and treat geography as flat text rather than a hierarchy [ 14 ] . Carefully curated benchmarks can transform fields in other domains: ClimSim ––a large-scale climate simulation dataset––unlocked hybrid physics–ML climate modelling [ 23 ] ; DecodingTrust ––a trustworthiness evaluation suite for frontier LLMs––exposed safety gaps in modern language models [ 20 ] ; the PRISM Alignment Dataset ––a cross-cultural human-feedback resource––broadened evaluation of alignment techniques across diverse regions [ 9 ] . Inspired by these successes, we release ICE-ID , the first large-scale open benchmark focused on long-term person matching in a national population. Spanning eight Icelandic censuses (1703–1920) and covering more than 220 years, ICE-ID contains 984 028 raw rows and 200 k high-confidence cluster labels created by domain experts. We formulate two tasks: 1. Intra-census linkage : identify the same individual within a single census. 2. Cross-census linkage : identify the same individual across successive censuses despite spelling drift and age progression. Train/dev/test splits follow a strictly temporal protocol: pre-1870 rows for training, 1870–1900 for validation, and 1901–1920 for held-out testing, mirroring real archival workflows. Evaluation combines pairwise metrics (precision, recall, F 1 , ROC-AUC) with clustering quality (Adjusted Rand Index, ARI). We benchmark four model families: • Deterministic rules : phonetic and location blocking with Jaro–Winkler and hierarchical filters; • Symbolic reasoning : a Non-Axiomatic Reasoning System (NARS) that learns identity rules from streaming examples; • Deep tabular networks : TabNet [ 1 ] , TabTransformer [ 8 ] , and FT-Transformer [ 6 ] , evaluated under distribution-shift benchmarks TableShift [ 5 ] and TabReD [ 17 ] . Our results confirm that transformer-style tabular models outperform classical heuristics on in-distribution data but lose up to 30% F 1 when evaluated on censuses 50 years after their training period, whereas NARS degrades more gracefully. These findings underline the need for hybrid pipelines that fuse symbolic and neural evidence when tackling deeply non-stationary historical data. 1.1 Related Work Identity resolution (also known as entity resolution or record linkage) aims to identify when two records refer to the same real-world entity. The foundational probabilistic model by Fellegi and Sunter formalizes matching decisions based on likelihood ratios of agreement patterns across fields [ 4 ] . Building on this, unsupervised methods such as ZeroER model match and non-match distributions via Gaussian Mixture Models and enforce transitivity constraints, achieving performance comparable to supervised learners without labeled data [ 21 ] . Supervised deep learning approaches have set state-of-the-art performance. Ditto serializes record pairs as text sequences and fine-tunes pre-trained Transformers, achieving state-of-the-art results on product matching benchmarks such as Abt–Buy [ 12 ] . Hierarchical Graph Attention Networks (HierGAT) incorporate both attribute-level and graph-level attention to enforce collective consistency, yielding top F 1 scores on standard ER datasets [ 22 ] . Hybrid rule-guided methods like GraphER leverage Graph Differential Dependencies to guide a graph neural network, offering interpretability alongside competitive performance in both graph-structured and relational entity resolution tasks [ 7 ] . On the unsupervised front, Bayesian graphical models treat linkage as a latent clustering problem under exchangeable random partition priors with realistic distortion processes, delivering robust performance without labels [ 13 ] . Zero-shot entity matching has been advanced by AnyMatch, which fine-tunes a small language model on synthetic matching examples; it reaches average F 1 within 4.4% of a GPT-4-based matcher while reducing inference cost by orders of magnitude [ 24 ] . Large Language Models (LLMs) have also been integrated into matching pipelines. BoostER uses GPT-4 as an on-demand oracle, selectively querying ambiguous pairs to refine match probabilities with minimal training effort [ 10 ] . Explanation-driven approaches recast matching as a conditional generation task, distilling LLM reasoning into smaller models and improving out-of-domain generalization [ 19 ] . The current frontier treats LLMs end-to-end for matching. MatchGPT employs GPT-4 with carefully designed prompts to achieve competitive F 1 on standard benchmarks, albeit at significant computational cost [ 16 ] . These developments illustrate the evolution from statistical foundations through supervised and unsupervised methods toward LLM-centric solutions, highlighting trade-offs among accuracy, generalization, and efficiency. Historical census linkage in demographic research has progressed from simple phonetic and geographic blocking techniques to large-scale supervised matchers. Early efforts applied deterministic string–matching rules (e.g., Soundex) and geographic blocking to reduce candidate comparisons. Feigenbaum et al. provide a comprehensive analysis of how training data quality affects census linkage, deploying supervised name and age similarity features calibrated via manual and crowd-sourced genealogies [ 3 ] . Ruggles et al. survey the progression of demographic ER methods in an Annual Review of Sociology, noting the incorporation of kinship networks and longitudinal residence trajectories [ 18 ] . Nevertheless, publicly available datasets that encode both household and inter-household kinship signals with multi-decadal links remain rare. Standard entity resolution benchmarks—such as product catalogs and citation graphs—typically span only short, modern timeframes and exhibit limited noise patterns. Papadakis et al. re-evaluated thirteen common ER datasets, showing most are easy classification tasks solvable by simple threshold rules and lacking hierarchical geographic or kinship structure [ 14 ] . To expose real-world robustness deficits, Gardner et al. proposed TableShift, a benchmark of fifteen tabular classification tasks with natural domain and temporal shifts [ 5 ] , and Rubachev et al. introduced TabReD, a suite of eight industry-grade datasets with explicit time-based train/test splits [ 17 ] . However, neither includes genealogical hierarchies or