OpenSanctions Pairs: Large-Scale Entity Matching with LLMs

Chandler Smith, Magnus Sesodia, Friedrich Lindenberg, Christian Schroeder de Witt · 2026

Read OpenSanctions Pairs: Large-Scale Entity Matching with LLMs by Chandler Smith, Magnus Sesodia, Friedrich Lindenberg, Christian Schroeder de Witt (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-11051.txt.

Introduction

Chandler Smith and coauthors release OpenSanctions Pairs, the first large-scale public benchmark for matching entities across sanctions and open-source intelligence data. It holds 755,540 expert-labeled pairs drawn from 293 source datasets across 45 jurisdictions, spanning many languages and writing systems with messy, time-varying records. Baselines show a production rule-based matcher over-matching while language models stumble on cross-script transliteration, with GPT-4o reaching 99.0 percent F1, suggesting pairwise matching nears its practical ceiling.

Catalog record

CREATORChandler Smith, Magnus Sesodia, Friedrich Lindenberg, Christian Schroeder de Witt
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE34543
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.11051

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

OpenSanctions Pairs: Large-Scale Entity Matching with LLMs Chandler Smith, Magnus Sesodia, Friedrich Lindenberg, Christian Schroeder de Witt Abstract We release OpenSanctions Pairs, a large-scale entity matching benchmark derived from real-world international sanctions aggregation and analyst deduplication. The dataset contains 755,540 labeled pairs spanning 293 heterogeneous sources across 31 countries, with multilingual and cross-script names, noisy and missing attributes, and set-valued fields typical of compliance workflows. We benchmark a production rule-based matcher (nomenklatura RegressionV1 algorithm) against open- and closed-source LLMs in zero- and few-shot settings. Off-the-shelf LLMs substantially outperform the production rule-based baseline (91.33% F1), reaching up to 98.95% F1 (GPT-4o) and 98.23% F1 with a locally deployable open model (DeepSeek-R1-Distill-Qwen-14B). DSPy MIPROv2 prompt optimization yields consistent but modest gains, while adding in-context examples provides little additional benefit and can degrade performance. Error analysis shows complementary failure modes: the rule-based system over-matches (high false positives), whereas LLMs primarily fail on cross-script transliteration and minor identifier/date inconsistencies. These results indicate that pairwise matching performance is approaching a practical ceiling in this setting, and motivate shifting effort toward pipeline components such as blocking, clustering, and uncertainty-aware review. Code available at https://github.com/chansmi/OSINT_entity_resolution † † affiliationnotice: Equal contribution 1 Introduction Entity matching (EM) determines whether two records refer to the same real-world entity. The task is also known as entity resolution, record linkage, field matching, and deduplication, with the latter typically referring to matching records within the same dataset and record linkage to matching across different datasets ( 9 ; 3 ) . EM is a core component of data integration pipelines, as matching errors propagate directly to downstream entity graph construction and analysis ( 9 ; 10 ) . Existing EM benchmarks are largely drawn from e-commerce and bibliographic domains and typically rely on a small number of datasets, often as few as two, with closely aligned schemas, fixed field sets, and relatively standardized entity representations ( 14 ; 15 ; 21 ; 31 ) . As a result, they fail to reflect deployment settings in which entity matching is performed across tens to hundreds of datasets with heterogeneous schemas, overlapping but non-identical fields, and substantial data quality issues such as missing values, inconsistencies, and noise. In these settings, entities may be represented using aliases or transliterations and appear in non-Latin scripts, e.g., Cyrillic or Chinese characters, making high-precision matching significantly more challenging. Sanctions data provides a politically and operationally important example of this setting. International authorities publish overlapping lists of sanctioned individuals and organizations, and financial institutions must screen customers against these lists to avoid regulatory penalties and reputational damage, with compliance failures resulting in fines of hundreds of millions of dollars ( 38 ) . Effective entity matching is critical for both regulatory compliance and detecting sanctions evasion, where individuals obscure ownership through shell companies and name variations across jurisdictions ( 19 ) . Despite this practical importance, sanctions data are typically proprietary or access-restricted, and to our knowledge no public benchmark for pairwise entity matching on sanctions data has been published. To address this gap, we present the first systematic study of pairwise EM techniques on large-scale sanctions data in collaboration with OpenSanctions 1 1 1 https://opensanctions.org . We aggregate 293 datasets from 31 countries, with human analysts performing pairwise matching ( 23 ) . From this process, we extract 755,540 labeled entity pairs covering over one million unique entities. Crucially, these labels reflect expert human decisions under incomplete evidence, mirroring real-world EM practice rather than definitive ground truth. Our experiments reveal that methods based on large language models (LLMs) achieve F1 scores up to 98.95%, compared to 91.33% for the legacy rule-based matcher previously used in production. This performance approaches observed human labeling consistency on the dataset, indicating diminishing returns from further optimization of pairwise matching models. We therefore argue that future work should prioritize other components of the entity resolution pipeline, including blocking, clustering, provenance modeling, and uncertainty handling ( 10 ; 4 ) . Our contributions: 1. We release and document a large-scale EM dataset derived from real sanctions OSINT data, an order of magnitude larger than existing benchmarks and substantially more heterogeneous. 2. We establish baselines comparing legacy rule-based matching with LLM-based approaches, demonstrating that LLMs reliably replicate expert decisions. 3. We analyze failure modes to identify where future research effort is most productive beyond pairwise matching. Table 1: Comparison with existing entity matching benchmarks. #Matches refers to the number of positive pairwise matches; * denotes deduplicated entities, not pairwise matches. Dataset #Entities #Matches #Sources #Unique Fields Domain Fodors-Zagat ( 20 ; 21 ) 946 110 2 6 Restaurant iTunes-Amazon ( 20 ; 21 ) 539 132 2 8 Music Walmart-Amazon ( 20 ; 21 ) 10,242 962 2 5 E-commerce Abt-Buy ( 34 ) 1,081 + 1,092 1,097 2 4 E-commerce Amazon-GoogleProducts ( 34 ) 1,363 + 3,226 1,300 2 4 E-commerce DBLP-ACM ( 34 ) 2,614 + 2,294 2,224 2 4 Bibliographic DBLP-Scholar ( 34 ) 2,616 + 64,263 5,347 2 4 Bibliographic WDC Products ( 31 ) 11,715 2,162 ∗ 3,259 5 E-commerce MusicBrainz20k ( 34 ) 19,375 16,250 5 10 Music OpenSanctions (ours) 1,002,093 581,149 293 131 Sanctions 2 Related Work Datasets A variety of datasets have been developed to evaluate entity matching methods. Standard benchmarks include product datasets such as Abt/Buy and Amazon/Google Products, as well as bibliographic datasets such as DBLP/ACM and DBLP/Scholar ( 34 ; 21 ; 14 ; 15 ) . These typically involve pairwise matching between two sources with clean, structured, and monolingual attributes. More recent benchmarks such as WDC Products increase scale and difficulty, but remain focused on e-commerce settings with limited source heterogeneity. Across domains, existing datasets are characterized by few sources, scalar attributes, and limited variation in scripts or provenance, and thus do not reflect the complexity of multi-source, multilingual environments. Our dataset differs along several dimensions. It operates in a sanctions OSINT setting with entities aggregated from 293 heterogeneous sources across 31 countries. Names and attributes span multiple scripts and transliteration standards, and entities are represented by set-valued and time-dependent attributes rather than single records. At 755,540 labeled pairs, it is substantially larger than most prior benchmarks. We provide a quantitative comparison in Table 1 . Entity Matching Methods EM originated in probabilistic formulations developed for administrative record linkage ( 9 ; 6 ) . Early work established the core pipeline of blocking, matching, and clustering, introducing methods to reduce candidate comparison costs ( 11 ; 10 ; 28 ; 29 ) . Neural approaches subsequently improved robustness to noisy and text-heavy data by learning attribute representations directly, with systems such as Magellan demonstrating gains over rule-based methods ( 13 ; 20 ) . Pretrained language models further advanced EM by encoding entire entity pairs as sequences and fine-tuning transformer-based classifiers ( 18 ; 27 ) . Parallel work has explored probabilistic and Bayesian formulations that explicitly model uncertainty and matching constraints ( 37 ; 35 ) . The most recent line of work applies large language models to entity matching via prompting, in-context learning, and fine-tuning, often achieving strong performance with minimal task-specific adaptation ( 22 ; 39 ; 30 ; 33 ; 7 ; 16 ; 17 ; 36 ; 40 )

Read the complete text