OmniMatch: A Large Language Model-Based Data Linkage Tool
Xiaowei Xu, Xingqiao Wang, Vivek Gunasekaran, Jonathan White, Anup Mathur · 2024
Read OmniMatch: A Large Language Model-Based Data Linkage Tool by Xiaowei Xu, Xingqiao Wang, Vivek Gunasekaran, Jonathan White, Anup Mathur (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/pmc-13177852.txt.
Description
Xiaowei Xu and four co-authors built a data linking tool on a language model in a 2024 paper. Linking records that name the same person or firm across files is a chore in health, census, and trade work. Files clash in tongue, form, and quality, with typos and gaps that break plain match rules and force costly hand checks.
Their tool, OmniMatch, tunes an open model to pair records across fields and tongues with no retraining for each new pair of files. It is built to shrug off noise, gaps, and mixed tongues. Since it runs on site, private data stays home, and the wild claims of cloud models are dodged. Tests in the paper span the hard cases the tool was built for.
Catalog record
| CREATOR | Xiaowei Xu, Xingqiao Wang, Vivek Gunasekaran, Jonathan White, Anup Mathur |
|---|---|
| YEAR | 2024 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 2 KB |
| VERSION | 1.0 |
| LICENSE | CC BY 4.0 |
| RIGHTS | CC BY |
| SOURCE | Europe PMC |