FUTURE: Flexible Unlearning for Tree Ensemble DOI: 10.1145/3746252.3760948 Conference: Proceedings of the 34th ACM International Conference on Information and Knowledge Management; November 10–14, 2025; Seoul, Republic of Korea. Proceedings of the 34th ACM International Conference on Information and
Ziheng Chen, Jin Huang, Jiali Cheng, Yuchan Guo, Mengjie Wang, Lalitesh Morishetti, Kaushiki Nag, Hadi Amiri · 2025
Read FUTURE: Flexible Unlearning for Tree Ensemble DOI: 10.1145/3746252.3760948 Conference: Proceedings of the 34th ACM International Conference on Information and Knowledge Management; November 10–14, 2025; Seoul, Republic of Korea. Proceedings of the 34th ACM International Conference on Information and by Ziheng Chen, Jin Huang, Jiali Cheng, Yuchan Guo, Mengjie Wang, Lalitesh Morishetti, Kaushiki Nag, Hadi Amiri (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-21181.txt.
Introduction
Ziheng Chen and co-authors present a forgetting method for tree ensemble models in a 2025 paper. Tree ensembles classify well in fields from finance to medicine. Privacy rules now grant a right to be forgotten, so models must drop sensitive training samples on request. Past forgetting tools fit one model type or lean on tree structure, which scales poorly.
Their method, FUTURE, turns forgetting into a gradient based optimization task. A probabilistic stand-in makes the non-smooth tree model trainable end to end. Tests on real data sets show strong and successful forgetting, they report. The approach aims to stay flexible across ensemble types.
Catalog record
| CREATOR | Ziheng Chen, Jin Huang, Jiali Cheng, Yuchan Guo, Mengjie Wang, Lalitesh Morishetti, Kaushiki Nag, Hadi Amiri |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 20 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2508-21181
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
FUTURE: Flexible Unlearning for Tree Ensemble DOI: 10.1145/3746252.3760948 Conference: Proceedings of the 34th ACM International Conference on Information and Knowledge Management; November 10–14, 2025; Seoul, Republic of Korea. Proceedings of the 34th ACM International Conference on Information and Knowledge Management (CIKM ’25), November 10–14, 2025, Seoul, Republic of Korea ISBN: 979-8-4007-2040-6/2025/11 DOI: 10.1145/3746252.3760948 ISBN: 978-1-4503-XXXX-X/2018/06 8928 CCS: Computing methodologies Learning paradigms CCS: Computing methodologies Learning settings CCS: Computing methodologies Neural networks CCS: Security and privacy Social network security and privacy Ziheng Chen, Jin Huang, Jiali Cheng, Yuchan Guo, Mengjie Wang, Lalitesh Morishetti, Kaushiki Nag, Hadi Amiri Abstract . Tree ensembles are widely recognized for their effectiveness in classification tasks, achieving state-of-the-art performance across diverse domains, including bioinformatics, finance, and medical diagnosis. With increasing emphasis on data privacy and the right to be forgotten , several unlearning algorithms have been proposed to enable tree ensembles to forget sensitive information. However, existing methods are often tailored to a particular model or rely on the discrete tree structure, making them difficult to generalize to complex ensembles and inefficient for large-scale datasets. To address these limitations, we propose FUTURE, a novel unlearning algorithm for tree ensembles. Specifically, we formulate the problem of forgetting samples as a gradient-based optimization task. In order to accommodate non-differentiability of tree ensembles, we adopt the probabilistic model approximations within the optimization framework. This enables end-to-end unlearning in an effective and efficient manner. Extensive experiments on real-world datasets show that FUTURE yields significant and successful unlearning performance. 1. Introduction Advances in machine learning have significantly contributed to progress in a wide range of domains. In particular, tree-based ensembles have emerged as a powerful approach for addressing classification tasks, especially in tabular data – fundamental for applications in finance and healthcare. However, the widespread adoption of tree-based ensembles raises concerns about privacy leakage ( Zhang et al. 2025 ; Liu et al. 2024 ; Gao et al. 2020 ) , as training data containing sensitive relationships can be implicitly “memorized” within model parameters. To mitigate the risk of misuse, recent regulatory policies have established the right to be forgotten ( Chen et al. 2024 ; Cheng and Amiri 2025a ) , allowing users to remove private data from online platforms. Consequently, a range of tree-based ensemble unlearning methods have been developed to effectively erase specific knowledge from a trained model without requiring full retraining ( Liu et al. 2025b ) . Machine unlearning ( Bourtoule et al. 2021 ; Liu et al. 2025b ; Fan et al. 2025 ; Jiali Cheng and Hadi Amiri 2025 ; Cheng and Amiri 2025c ; Huang et al. 2025 ; Liu et al. 2025a ; Cheng and Amiri 2025b ) for tree-based ensembles has recently gained considerable attention. Several works, including ( Brophy and Lowd 2021 ) and ( Schelter et al. 2021 ) , focus specifically on unlearning in Random Forests(RF). ( Brophy and Lowd 2021 ) selectively adjusts subtrees by modifying only the necessary upper-level split nodes. In contrast, ( Schelter et al. 2021 ) introduces a robustness quantification factor to identify splits that remain stable when a limited number of training instances are removed, enabling low-latency unlearning. ( Lin et al. 2023 ) and ( Wu et al. 2023 ) are proposed for unlearning in Gradient Boosting Decision Trees (GBDT). Both algorithms exploit the internal tree structure and updates the threshold by incremental calculation in split gains and derivatives, resulting in a high computational cost for unlearning individual instances. In particular, ( Lin et al. 2023 ) assumes that less than 1\% of the data will be removed, as the method becomes time-consuming for larger removal requests. In short, existing methods require full access to the models’ training mechanisms and only work with specific tree-based ensembles. Moreover, their reliance on discrete structures hinders gradient-based updates and makes them scale poorly with the size of the forgetting set and the model. In this work, we proposed FUTURE, an approximate unlearning algorithm designed to operate on any tree-based ensemble classifier. Inspired by ( Irsoy et al. 2012 ; Dong et al. 2021 ; Liu et al. 2024 ) , we first approximate the tree ensemble using a soft decision forest ( Kontschieder et al. 2015 ; Lucic et al. 2022 ; Luo et al. 2021 ) , replacing each split with a differentiable sigmoid function. This surrogate is trained end-to-end for the unlearning task, and the learned thresholds are ultimately transferred back to update the original ensemble. Overall, the main contributions of our work are as follows: • We unveil the limited generality and poor scalability of existing methods, and propose FUTURE, the first model-agnostic unlearning algorithm designed for tree ensembles. • FUTURE is an end-to-end unlearning framework for tree ensembles by leveraging a probabilistic approximation of their discrete structure, enabling scalability with respect to both ensemble size and forgetting set. • We conduct experiments on several real datasets, demonstrating that FUTURE can effectively and efficiently remove data as well as maintaining 95% predictive power on test set, even when deleting 40% of data. 2. Background and Preliminaries 2.1. Machine Unlearning In this work, we focus on machine unlearning for classification. This task involves selectively removing specific instances from a well-trained tree ensemble g without retraining the entire model. We denote the set of samples to forget as \mathcal{D}_{f}\subset\mathcal{D} . The set of samples to be retained is denoted as \mathcal{D}_{r} , with the assumption that \mathcal{D}_{f}\cup\mathcal{D}_{r}=\mathcal{D} and \mathcal{D}_{f}\cap\mathcal{D}_{r}=\emptyset ( Cheng and Amiri 2024 ; Chen et al. 2025 ; Gao 2024 ) . The goal of machine unlearning is to obtain a new model g^{u} that removes the information contained in \mathcal{D}_{f} from g without adversely affecting its performance on \mathcal{D}_{r} ( Zhang et al. 2021 ) . Given the time-consuming nature of fully retraining the model on \mathcal{D}_{r} to obtain an retrained model g^{*} , our target is to approximate g^{*} by evolving g using \mathcal{D}_{f} as follows: g\xrightarrow{\mathcal{D}_{f}}g^{u}\simeq g^{*} 2.2. Tree-Based Ensembles Figure 1. Approximating Tree-Based Ensembles With Soft Decision Forest. Left: Original Tree Ensembles \mathcal{T} . Right: Soft Decision Forest \tilde{\mathcal{T}} The tree ensemble g can be viewed as a set of trees with weights w_{i} . For any sample \mathbf{x} , the tree ensemble return the weighted vote of its trees (1) g(y|\mathbf{x})=\arg\max\limits_{y}\sum^{M}_{i=1}w_{i}\cdot\mathcal{T}_{i}(y|\mathbf{x}), where \mathcal{T}_{i}:\mathcal{X}\mapsto\mathcal{Y} is a tree-structured classifier consisting of decision nodes and leaf nodes. As shown in figure 1 , for one sample \mathbf{x} , a node j is activated if its parent node p_{j} is activated and feature \mathbf{x}_{f_{j}} satisfies the threshold condition determined by \theta_{j} . Note that the root node is always activated. Let t_{j}(\mathbf{x)} indicate if node j is activated: (2) t_{j}(\mathbf{x})=\begin{cases}1,&\text{if }j\text{ is a root node}\\ t_{p_{j}}(\mathbf{x})\cdot\mathbb{I}[\mathbf{x}_{f_{j}}\geq\theta_{j}],&\text{if }j\text{ is a right child of }p_{j}\\ t_{p_{j}}(\mathbf{x})\cdot\mathbb{I}[\mathbf{x}_{f_{j}}<\theta_{j}],&\text{if }j\text{ is a left child of }p_{j}\end{cases} For an instance \mathbf{x} , it traverses from the root through activated decision nodes and ultimately reaches a single leaf node. Each leaf node j is associated with a prediction distribution t_{l}(y\mid j) that determines the output of \mathcal{T} for \mathbf{x} . Let \mathcal{N}_{l} denotes the set of leaf nodes in \mathcal{T} , then we have: (3) \mathcal{T}(y|\mathbf{x})=\sum_{j\in\mathcal{N}_{l}}t_{j}(\mathbf{x})\cdot t_{l}(y|j) In practice, all samples reaching the same leaf node are assigned