Reproducibility Companion Paper: Making Users Indistinguishable: Attribute-wise Unlearning in Recommender Systems CCS: Information systems Recommender systems CCS: Information systems Collaborative filtering CCS: Security and privacy Social network security and privacy
Yuyuan Li, Junjie Fang, Chaochao Chen, Xiaolin Zheng, Yizhao Zhang, Zhongxuan Han · 2025
Read Reproducibility Companion Paper: Making Users Indistinguishable: Attribute-wise Unlearning in Recommender Systems CCS: Information systems Recommender systems CCS: Information systems Collaborative filtering CCS: Security and privacy Social network security and privacy by Yuyuan Li, Junjie Fang, Chaochao Chen, Xiaolin Zheng, Yizhao Zhang, Zhongxuan Han (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-23032.txt.
Introduction
Yuyuan Li and co-authors publish a companion paper that lets others repeat their earlier study, in a 2025 paper. The original work made users of recommendation systems harder to single out. It did so by unlearning traits of each user, one trait at a time. The method was first shown at a major multimedia conference.
The companion paper aims to validate the method and help others copy the results. It describes the cleaned data sets in detail. It also lays out the code structure and the settings files. The test environment is spelled out, and the repeated results are reported. Reproducibility papers of this kind let a field check claims instead of taking them on trust.
Catalog record
| CREATOR | Yuyuan Li, Junjie Fang, Chaochao Chen, Xiaolin Zheng, Yizhao Zhang, Zhongxuan Han |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 13 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2503-23032
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Reproducibility Companion Paper: Making Users Indistinguishable: Attribute-wise Unlearning in Recommender Systems CCS: Information systems Recommender systems CCS: Information systems Collaborative filtering CCS: Security and privacy Social network security and privacy Yuyuan Li, Junjie Fang, Chaochao Chen, Xiaolin Zheng, Yizhao Zhang, Zhongxuan Han Abstract . In this paper, we reproduce the experimental results presented in our previous work titled “Making Users Indistinguishable: Attribute-wise Unlearning in Recommender Systems,” which was published in the proceedings of the 31st ACM International Conference on Multimedia. This paper aims to validate the effectiveness of our proposed method and help others reproduce our experimental results. We provide detailed descriptions of our preprocessed datasets, source code structure, configuration file settings, experimental environment, and reproduced experimental results. 1. Contribution summary To protect the sensitive attribute of users in recommender systems, Attribute Unlearning (AU) aims to make target attributes indistinguishable from attackers ( Li et al. 2024 ) . we investigate a strict and practical setting of AU, namely Post-Training Attribute Unlearning (PoT-AU), where unlearning can only be performed after the recommendation model completes training ( Chen et al. 2024 ; Feng et al. 2025 ) . To address the PoT-AU problem, we design a two-component loss function to balance the performance of recommendation and unlearning ( Li et al. 2023 ) . Specifically, we investigate two types of distinguishability measurements, i.e., User-to-User (U2U) and Distribution-to-Distribution (D2D). We conduct experiments to validate the effectiveness of our proposed methods. 2. ARTIFACTS DESCRIPTION 2.1. Dataset Preparation Experiments are conducted on three real word datasets. MovieLens 100K (ML-100K) and 1M (ML-1M) 1 1 1 https://grouplens.org/datasets/movielens/ are publicly available. The LFM-2B 2 2 2 http://www.cp.jku.at/datasets/LFM-2b dataset is currently not publicly available due to copyright restrictions. However, we provide the raw data that we previously downloaded. We filter out the users and items under the minimum interaction threshold. Specifically, it is set to 5 for both ML-100K and ML-1M, and 120 for LFM-2B, respectively. We adopt a leave-one-out evaluation, where each user’s most recent interaction and 99 sampled negative interactions are used as test samples, with the remaining interactions serving as training samples. The complete source code, including datasets, is available on both Baidu Netdisk and Google Drive: https://pan.baidu.com/s/1Clq7_lFf5D1Di_y4pJKkLA?pwd=bqru https://drive.google.com/file/d/1Ffe7Vv4pI4Icz2vyj2PqUtSUTBfGVWOl/view For each dataset, we provide raw files, preprocessed files, and a data processing program (data_process.ipynb). The preprocessed files are as follows: • pos_dict.npy : A dictionary that maps the user IDs to their positive item IDs. • s_pre_adj_mat.npz : A sparse adjacency matrix that defines the graph structure for the LightGCN model. Automatically generated when running the LightGCN model. • test_negative_ratings : The testing dataset contains one positive item per user and 99 sampled negative items. • train_ratings : The training dataset containing user IDs, item IDs, and ratings. • user_gender.npy : A dictionary that maps the user IDs to their gender. 2.2. Source Code Structure Our source codes consists of 6 folders, 2 main program files, and 1 Jupyter notebook located in the root directory. Github repository: https://github.com/oktton/Attribute-wise-Unlearning . • Folders: – data/ : contains routines for loading and datasets. – models/ : includes source code for recommendation models. – methods/ : stores different unlearning methods. – utils/ : contains utility functions for model preparation, evaluation, and saving. – configs/ : includes configuration files, i.e., “exp_config.json” and “attack_config.json”. “exp_config.json” is for training experiments and “attack_config.json” is for attack experiments. – exp_results/ : stores experimental results, including log files and model files, which will be automatically generated after running experiments. • Files: – main.py : the main program for unlearning experiments. – attack.py : the main program for attacking experiments. – generate_embeding.ipynb : A Jupyter notebook for generating attribute distribution plots of user embeddings. The parameters in “exp_config.json” are defined as follows, while other parameters remain unchanged. • method : The unlearning method, selected from {“original”, “u2u”, “d2d”, “retrain”, “adv”}. • model : The recommendation model architecture, selected from {“ncf”, “lgcn”}. • dataset : The name of dataset used in the experiment, selected from {“ml-100k”, “ml-1m”, “lfm-2b”}. • device : The computing device used for training, e.g., “cuda:0”. • au_trade_off : The weight coefficient for regularization loss in optimization objective of u2u or d2d. Default is 1e-6. • retrain_trade_off : The weight coefficient for distinguishability loss in optimization objective of retrain method. Default is 1. The parameters in “attack_config.json” are similar to those in the “exp_config.json”, while other parameters remain unchanged. 2.3. Experimental Environment Our source codes are tested in the following environment. • System and Hardware : Ubuntu 18.04.5 LTS, Intel(R) Xeon(R) Silver 4114 CPU @ 2.20GHz and NVIDIA GeForce RTX 4080. • CUDA Toolkit and CuDNN : Tested with CUDA==12.4 and CuDNN==8.9.3. • Version and Dependencies : The main libraries and their versions used in this experiment are listed below: python==3.10.9 numpy==1.24.3 torch==2.6.0+cu118 xgboost==2.1.2 scikit-learn==1.5.2 pandas==2.2.3 scipy==1.14.1 tqdm==4.66.6 We have packaged them in a file named “requirements.txt” in the root directory. You can install the dependencies by “pip install -r requirements.txt”. 3. REPRODUCTION DETAILS In this section, we provide detailed instructions on how to reproduce our experiments. All the parameters have been set and the codes do not need to be modified. The specific meanings of these parameters were introduced in the section 2 . you can run the following command to reproduce training experiments: ⬇ cd Attribute - wise - Unlearning python main . py After the training is complete, the results will be stored in the exp_results folder, including model files and training logs. Then run the following command to reproduce attack experiments: ⬇ python attack . py Finally, the attack logs of the attack experiment will also be stored in the exp_results folder. 4. EVALUATION EXPERIMENTS In this section, we introduce experimental setup and results. We study the effectiveness (i.e., recommendation performance, unlearning performance, and efficiency) of our proposed two attribute unlearning methods. Additionally, we analyze the changes in the distribution of user embeddings before and after unlearning to elucidate the mechanisms underlying our proposed methods. 4.1. Experimental Setup For recommendation models, we adopt NMF ( He et al. 2017 ) and LightGCN ( He et al. 2020 ) as the base models. To evaluate the unlearning performance, we employ two types of attackers: MLP ( Gardner and Dorling 1998 ) and XGB ( Chen et al. 2015 ) . We compare our proposed unlearning methods (U2U-R and D2D-R) with representative baselines, including Original (without unlearning), Retrain ( Zafar et al. 2019 ) , and Adv-InT ( Ganhör et al. 2022 ) . 4.2. Evaluation Results 4.2.1. Unlearning Performance We use the performance of attackers to evaluate the unlearning performance. We use Accuracy (Acc), Precision, Recall, and Area Under Curve (AUC) as evaluation metrics. Table 3 shows the results of unlearning performance. 4.2.2. Recommendation Performance We use Normalized Discounted Cumulative Gain (NDCG) and Hit Ratio (HR) as evaluation metrics. For both metrics, we report the results with truncation values K = 5 and 10. Table 1 shows the results of recommendation performance. Table 1. Results of recommendation performance. The top results are highlighted in bold. NMF NDCG@5 HR@5 NDCG@10 HR@10 ML-100K Original 0.3083 0.4532 0.3687 0.6353 U2U-R 0.2444 0.3545 0.2900 0.4989 D2D-R 0.3013 0.4393 0.3630 0.6330 Retrain 0.3180 0.4579 0.3767 0.6374 Adv-InT 0.3120 0.4569 0.3650 0.6246 ML-1M Original 0.3695 0.5318