[Experiments & Analysis] Evaluating Joinable Column Discovery Approaches for Context-Aware Search Conference: ; ;
Harsha Kokel, Aamod Khatiwada, Tejaswini Pedapati, Haritha Ananthakrishnan, Oktie Hassanzadeh, Horst Samulowitz, Kavitha Srinivas · 2025
Read [Experiments & Analysis] Evaluating Joinable Column Discovery Approaches for Context-Aware Search Conference: ; ; by Harsha Kokel, Aamod Khatiwada, Tejaswini Pedapati, Haritha Ananthakrishnan, Oktie Hassanzadeh, Horst Samulowitz, Kavitha Srinivas (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-24599.txt.
Introduction
Harsha Kokel leads a team of seven authors on finding joinable columns. Two table columns are joinable if their values can link the tables. Automated analysis needs such links found without human help. Past methods used value overlap or meaning similarity alone. Which method wins for which data was poorly understood.
The team compare syntactic and semantic methods on seven benchmarks. The benchmarks span relational databases and loose data lakes. Six criteria are tested, from shared values to metadata meaning. Ensemble ranking, a blend of criteria, beat single-criterion methods. Meaning of values and metadata mattered most for data lakes. Size-based criteria mattered more for relational databases. The study gives builders clear rules for choosing a method.
Catalog record
| CREATOR | Harsha Kokel, Aamod Khatiwada, Tejaswini Pedapati, Haritha Ananthakrishnan, Oktie Hassanzadeh, Horst Samulowitz, Kavitha Srinivas |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 53 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2510-24599
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
[Experiments & Analysis] Evaluating Joinable Column Discovery Approaches for Context-Aware Search Conference: ; ; Harsha Kokel, Aamod Khatiwada, Tejaswini Pedapati, Haritha Ananthakrishnan, Oktie Hassanzadeh, Horst Samulowitz, Kavitha Srinivas Abstract . Joinable Column Discovery is a critical challenge in automating enterprise data analysis. While existing approaches focus on syntactic overlap and semantic similarity, there remains limited understanding of which methods perform best for different data characteristics and how multiple criteria influence discovery effectiveness. We present a comprehensive experimental evaluation of joinable column discovery methods across diverse scenarios. Our study compares syntactic and semantic techniques on seven benchmarks covering relational databases and data lakes. We analyze six key criteria—unique values, intersection size, join size, reverse join size, value semantics, and metadata semantics—and examine how combining them through ensemble ranking affects performance. Our analysis reveals differences in method behavior across data contexts and highlights the benefits of integrating multiple criteria for robust join discovery. We provide empirical evidence on when each criterion matters, compare pre-trained embedding models for semantic joins, and offer practical guidelines for selecting suitable methods based on dataset characteristics. Our findings show that metadata and value semantics are crucial for data lakes, size-based criteria play a stronger role in relational databases, and ensemble approaches consistently outperform single-criterion methods. 1. Introduction With the rapid growth of enterprise data repositories, automating data discovery has become a central challenge in enabling scalable analytics. Data discovery refers to the process of identifying relevant data sources that can answer specific analytical or business questions. In the context of enterprise applications and natural language interfaces for data analysis ( WeideleRBVBCMMA23 ; Mihindukulasooriya23 ; WeideleMVRSFAMA24 ) , a key component of this process is the discovery of joinable tables : given a query table and a designated join column, determine which other tables and columns can be joined to enrich the query table with additional attributes and insights ( ALITE_khatiwadaSGM22 ) . Traditional approaches to joinable table discovery primarily rely on measuring similarity between columns. Two columns are considered syntactically joinable when they share overlapping values ( JOSIE_ZhuDNM19 ; LSHENSEMBLE_ZhuNPM16 ) , and semantically joinable when their embeddings are close in a semantic vector space ( PEXESO_DongT0O21 ; DeepJoin_Dong0NEO23 ; WarpGate_CongGFJD23 ) . While these approaches have shown promising results, they also exhibit significant limitations when applied to heterogeneous enterprise data lakes that contain tables from diverse and often unrelated domains. In such settings, naive similarity-based joins may lead to numerous false positives—joins that are statistically similar but contextually irrelevant—thereby reducing both precision and interpretability of downstream analyses. Despite several recent advances in data discovery and table search ( DBLP:conf/icde/SantosBMF22 ; DBLP:journals/pvldb/DengCCYCYSWLCJZJZWYWT24 ; DBLP:journals/corr/abs-2310-02656 ; DBLP:journals/tkde/GuoMHCG25 ; ChristensenLeventidisLissandriniDiRoccoMillerHose2025 ) , there remains limited understanding of when particular methods perform well, why they fail, and how different similarity criteria influence the effectiveness of join discovery. While these recent systems have introduced new models, indices, and benchmarks for large-scale or semantic data discovery, they typically focus on algorithmic innovation rather than comparative insight. In this paper, we take an empirical and analytical perspective, conducting a systematic experimental study of the criteria used for joinable column discovery to provide a clear, data-driven assessment of the accuracy of various methods across diverse settings. To this end, we consider both syntactic methods (e.g., LSH Ensemble ( LSHENSEMBLE_ZhuNPM16 ) , JOSIE ( JOSIE_ZhuDNM19 ) ) and semantic methods (e.g., DeepJoin ( DeepJoin_Dong0NEO23 ) , WarpGate ( WarpGate_CongGFJD23 ) ), and analyze how their performance varies under different data characteristics. We identify six key criteria that govern join effectiveness— unique values, intersection size, join size, reverse join size, value semantics, and metadata semantics —and study how each contributes to identifying meaningful joins. We further explore how combining these criteria through ensemble ranking can balance trade-offs between precision and recall, yielding more robust behavior across domains. Our evaluation spans seven benchmarks covering relational databases, enterprise data lakes, and fuzzy-join scenarios. The collection includes both established datasets from prior work and newly curated benchmarks, including manually annotated open data tables designed to capture context-aware joinability. This diversity allows us to assess method behavior across controlled and realistic conditions. Our study leads to several practical insights. We observe that no single approach dominates across all settings; metadata and value semantics emerge as the most decisive factors for heterogeneous data lakes, while size-based criteria play a stronger role in structured relational databases. Ensemble approaches that integrate multiple criteria consistently outperform individual measures, demonstrating the importance of multi-faceted evaluation in joinable column discovery. In summary, the contributions of this paper are as follows: • We conduct the first comprehensive experimental analysis of the accuracy of joinable column discovery methods across relational, data lake, and fuzzy-join scenarios. • We introduce a systematic framework for evaluating six key criteria that influence joinability and analyze their individual and combined impact on discovery effectiveness. • We curate and release seven benchmarks, including both prior and newly annotated datasets, to facilitate reproducible and context-aware evaluation. • We provide empirical guidelines for selecting appropriate methods and criteria based on dataset characteristics and analytical objectives. 2. Task Definition and Joinability Criteria Discovering joinable tables is a fundamental step in data discovery and integration, enabling users to enrich a query table with related attributes from other tables. Understanding what makes two columns joinable, and under what conditions certain definitions of joinability are more effective, is central to our analysis. In this section, we formalize the task of joinable column discovery and describe the different factors—or criteria—that influence whether two columns should be considered joinable in practice. These definitions and criteria form the conceptual foundation for our experimental study. 2.1. Joinable Column Discovery Task Two tables can be joined if each of them contains a column that can be joined. Consequently, the task of discovering joinable tables can be formulated as a joinable column search problem defined as follows: Definition 2.1. Joinable Column Search Problem: Given a collection of columns \mathcal{C} , a query column q_{Q} from a table Q , and a constant k , find top- k columns from the collection that are “joinable” to the query column q_{Q} . Various notions of “joinable” columns have been explored in the literature ( datalake_tutorial_NargesianZMPA19 ) . Popular definitions are based on set similarity, which measures set intersection size ( JOSIE_ZhuDNM19 ) , Jaccard similarity ( Mannheim_LehmbergRRMPB15 ) , or set containment ( LSHENSEMBLE_ZhuNPM16 ) between a pair of columns. Approaches that use such syntactic join measures are often referred to as equality join or equi-join approaches. Recent work has also explored the notion of semantically joinable . Specifically, Dong et al. ( PEXESO_DongT0O21 ) defined semantic joinability as a measure of similarity between the embeddings of values in the column (referred to as vector matching). Cong et al. ( WarpGate_CongGFJD23 ) studied semantic column joinability defined as similarity between column embeddings. Koutras et al. ( Valentine_KoutrasSIPBFLBK21 ) defined column pairs as semantically joinable if the columns share semantically equivalent instances. Notably, none of these approaches consider table context in the joinable column definition. In our study, we focus on joinable columns that are meaningful within their broader table context and refer to these as context-aware joinable columns . Definition 2.2. Context-aware Joinable Columns: Given a query column