GPT-HTree: A Decision Tree Framework Integrating Hierarchical Clustering and Large Language Models for Explainable Classification
Te Pei, Fuat Alican, Aaron Ontoyin Yin, Yiğit Ihlamur · 2025
Read GPT-HTree: A Decision Tree Framework Integrating Hierarchical Clustering and Large Language Models for Explainable Classification by Te Pei, Fuat Alican, Aaron Ontoyin Yin, Yiğit Ihlamur (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2501-13743.txt.
Introduction
Te Pei and co-authors present GPT-HTree, a 2025 framework for classification that stays readable to humans. Many accurate models are black boxes, which blocks their use where reasons matter. Their design chains three tools. First, hierarchical clustering groups people by their most telling features. Resampling then balances the classes inside each group. Decision trees finally draw a tailored sorting path for each cluster.
A large language model writes plain language descriptions of each cluster. Those notes bridge the numbers and the actions a user might take. The authors argue the chain keeps the accuracy of modern methods while restoring a clear chain of reasons. Each prediction can be traced through its cluster and its tree path. They present the framework as a template for explainable classification in data rich fields.
Catalog record
| CREATOR | Te Pei, Fuat Alican, Aaron Ontoyin Yin, Yiğit Ihlamur |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 37 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2501-13743
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
GPT-HTree: A Decision Tree Framework Integrating Hierarchical Clustering and Large Language Models for Explainable Classification Te Pei, Fuat Alican, Aaron Ontoyin Yin, Yiğit Ihlamur Abstract Traditional decision trees often fail on heterogeneous datasets, overlooking differences among diverse user segments. This paper introduces GPT-HTree , a framework combining hierarchical clustering, decision trees, and large language models (LLMs) to address this challenge. By leveraging hierarchical clustering to segment individuals based on salient features, resampling techniques to balance class distributions, and decision trees to tailor classification paths within each cluster, GPT-HTree ensures both accuracy and interpretability. LLMs enhance the framework by generating human-readable cluster descriptions, bridging quantitative analysis with actionable insights. Applied to venture capital, where the random success rate of startups is 1.9%, GPT-HTree identifies explainable clusters with success probabilities up to 9 times higher. One such cluster, serial-exit founders , comprises entrepreneurs with a track record of successful acquisitions or IPOs is 22x more likely to succeed compared to early professionals . 1 Introduction Decision trees are fundamental tools in machine learning (ML), prized for their interpretability and simplicity in classification tasks. By providing clear decision paths, they enable users to understand and trust the reasoning behind predictions. However, their effectiveness diminishes when applied to heterogeneous datasets comprising entities with varying characteristics. Uniform decision paths often fail to account for the nuanced differences among diverse segments, leading to oversimplified or misleading classifications. Unsupervised clustering methods, on the other hand, excel in discovering latent structures within complex datasets. These methods, including hierarchical clustering, k-means, and DBSCAN, are powerful tools for segmenting populations into meaningful clusters without requiring predefined labels. While they are effective for uncovering hidden patterns, their primary drawback is a lack of explainability. Clusters produced by unsupervised methods often lack intuitive descriptions or actionable insights, making it difficult to interpret their relevance or apply them in practical decision-making scenarios. 1.1 Motivation The primary motivation for this work stems from three key observations: • Traditional decision trees struggle with heterogeneous populations, often producing oversimplified or misleading classifications. • Current approaches lack mechanisms to adapt decision paths to the unique characteristics of different subgroups within a dataset, leading to generalized rules that fail to capture the nuanced differences among diverse populations. • There is an increasing demand for classification methods that balance interpretability and accuracy, particularly in handling diverse populations. Leveraging the reasoning capabilities of LLMs present a potential solution to address this need. Explainability is particularly critical in high-stakes domains such as venture capital, healthcare, and criminal justice. Supervised learning methods offer interpretability but rely on exhaustive labeled data and predefined categories. This reliance limits their ability to explore unknown or dynamic structures in data. In contrast, unsupervised methods are powerful for uncovering novel patterns but fail to provide the transparency required for practical applications. Achieving explainable cluster creation is essential for making informed decisions and deriving actionable insights, particularly in venture capital. 1.2 Empirical Context and Challenges in Venture Capital The domain of venture capital provides a compelling empirical study for the importance of explainable unsupervised clustering. At the inception stage, the probability of a startup achieving random success—defined as becoming a unicorn or reaching significant exit milestones—is approximately 1.9% [ 1 ] . However, empirical evidence shows that there are clusters of startups with significantly higher probabilities of success, up to 20%. Identifying these high-potential clusters is critical for venture capitalists, as it allows them to allocate resources and maximize investment outcomes. Using purely supervised methods to identify these clusters is impractical due to the infeasibility of exhaustively labeling all potential categories of startups. Moreover, the optimal number and structure of clusters are not known a priori, further limiting the scope of supervised approaches. While unsupervised methods can reveal these high-potential clusters, their lack of explainability prevents actionable decision-making. Venture capitalists require interpretable clusters to focus on the right channels to source the highest probability startups. To address these challenges, this work introduces GPT-HTree , an advanced framework that integrates hierarchical clustering with decision trees and large language models (LLMs). This hybrid approach leverages unsupervised clustering to uncover latent structures in the data while employing decision trees to provide localized, interpretable decision paths within each cluster. LLMs enhance the framework by generating human-readable descriptions of clusters, bridging the gap between quantitative analysis and qualitative insights. By enabling explainable cluster creation and classification, GPT-HTree empowers venture capitalists to identify and target the most promising clusters for deal sourcing and investment. This framework demonstrates empirical effectiveness by uncovering clusters that outperform the random success rate by 9x, a significant jump from the baseline probability of 1.9% to 17.4% and providing actionable insights into the determinants of entrepreneurial success. In doing so, GPT-HTree not only addresses the limitations of traditional decision trees and unsupervised methods but also sets a new direction for interpretable ML by integrating LLMs into the process. 1.3 Key Contributions Our work makes the following contributions: • A decision tree framework that integrates hierarchical clustering and LLMs to create personalized classification paths for different user segments • A new method using LLMs to automatically generate interpretable persona descriptions from quantitative cluster features, enabling human-interpretablity • An end-to-end pipeline that can take new founder profiles as input and generate both interpretable persona descriptions and probability estimates of success 2 Related Work There has been a growing body of research exploring the integration of ML and LLMs, a rapidly expanding area that holds the potential to advance the field toward artificial superintelligence (ASI). This emerging paradigm leverages LLMs for their reasoning and generative capabilities to perform ML processes such as data wrangling, feature modeling, and parameter optimization—tasks that have traditionally relied on human expertise. By automating these processes, LLMs not only reduce manual effort but also open new possibilities for more efficient and scalable workflows. Our work builds upon these advancements by demonstrating how LLMs can guide and enhance these traditionally human-driven tasks within the context of decision tree and clustering frameworks. Figure 1: An overview of the Machine Learning Workflow, where task specification serves as the input, encompassing key stages of data and feature engineering, model selection and hyperparameter optimization, and workflow evaluation. Figure reproduced from [ 2 ] . 2.1 Decision Trees, Ensemble Methods, and Their Limitations Decision trees and their ensemble variants (e.g., Random Forests, Gradient Boosting) have long been favored for their interpretability and effectiveness in structured, relatively uniform datasets [ 3 , 4 , 5 , 6 , 7 , 8 , 9 ] . Despite their success in fields like text mining [ 10 ] , speech processing [ 11 , 12 ] , bioinformatics [ 13 ] , and web intelligence [ 14 , 15 ] , these methods face growing challenges as data becomes increasingly large, heterogeneous, and multimodal. Scaling such models often involves labor-intensive steps—feature engineering, hyperparameter tuning, and iterative refinement—making it difficult to adapt swiftly to complex, evolving task requirements. 2.2 Clustering-Based Classification and Hybrid Approaches Clustering-based classification represents an effort to better handle complex data distributions by partitioning instances into meaningful subgroups before applying specialized models within each cluster. This strategy can enhance both model accuracy and interpretability, yet constructing and maintaining such pipelines remains time-consuming and knowledge-intensive. The need