Accurate and Diverse Recommendations via Propensity-Weighted Linear Autoencoders Conference: Proceedings of the 2025 Annual International ACM SIGIR Conference on Research and Development in Information Retrieval in the Asia Pacific Region; December 7–10, 2025; Xi’an, China Proceedings of the 2025 An

Kazuma Onishi, Katsuhiko Hayashi, Hidetaka Kamigaito · 2025

Read Accurate and Diverse Recommendations via Propensity-Weighted Linear Autoencoders Conference: Proceedings of the 2025 Annual International ACM SIGIR Conference on Research and Development in Information Retrieval in the Asia Pacific Region; December 7–10, 2025; Xi’an, China Proceedings of the 2025 An by Kazuma Onishi, Katsuhiko Hayashi, Hidetaka Kamigaito (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2512-20896.txt.

Introduction

Kazuma Onishi, Katsuhiko Hayashi, and Hidetaka Kamigaito fight popularity bias in recommendation lists. Popular items collect more clicks simply because more people see them, so training data skews toward hits. Lists then grow repetitive, and standard corrections punish popular items so hard that good recommendations suffer.

Their fix reshapes the correction score with a smooth sigmoid curve applied to log frequency. The gentler penalty frees room for less-seen items without burying genuine hits. Built into a linear autoencoder, the method sharply improved list diversity in experiments. Accuracy held steady at the same time, a pairing recommender designers rarely get from bias corrections.

Catalog record

CREATORKazuma Onishi, Katsuhiko Hayashi, Hidetaka Kamigaito
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE32 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2512-20896

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Accurate and Diverse Recommendations via Propensity-Weighted Linear Autoencoders Conference: Proceedings of the 2025 Annual International ACM SIGIR Conference on Research and Development in Information Retrieval in the Asia Pacific Region; December 7–10, 2025; Xi’an, China Proceedings of the 2025 Annual International ACM SIGIR Conference on Research and Development in Information Retrieval in the Asia Pacific Region (SIGIR-AP 2025), December 7–10, 2025, Xi’an, China DOI: 10.1145/3767695.3769512 ISBN: 979-8-4007-2218-9/2025/12 CCS: Information systems Recommender systems Kazuma Onishi, Katsuhiko Hayashi, Hidetaka Kamigaito Abstract . In real-world recommender systems, user-item interactions are Missing Not At Random (MNAR), as interactions with popular items are more frequently observed than those with less popular ones. Missing observations shift recommendations toward frequently interacted items, which reduces the diversity of the recommendation list. To alleviate this problem, Inverse Propensity Scoring (IPS) is widely used and commonly models propensities based on a power-law function of item interaction frequency. However, we found that such power-law-based correction overly penalizes popular items and harms their recommendation performance. We address this issue by redefining the propensity score to allow broader item recommendation without excessively penalizing popular items. The proposed score is formulated by applying a sigmoid function to the logarithm of the item observation frequency, maintaining the simplicity of power-law scoring while allowing for more flexible adjustment. Furthermore, we incorporate the redefined propensity score into a linear autoencoder model, which tends to favor popular items, and evaluate its effectiveness. Experimental results revealed that our method substantially improves the diversity of items in the recommendation list without sacrificing recommendation accuracy. The source code of our experiments is available on GitHub at https://github.com/cars1015/IPS-LAE . † † cc-license: by 1. Introduction In today’s world, users face an overwhelming number of products such as movies and music. Recommender systems have become essential for helping users discover preferred items and supporting various services. In recent years, implicit feedback recommendation ( Hu et al. 2008 ) , which relies on feedback such as clicks, views, and purchases, has been widely adopted in practice. The data used in these systems is biased toward popular items, and user feedback also tends to concentrate on a small subset of items ( Zhao et al. 2023 ; Abdollahpouri and Mansoury 2020 ) . As a result, item frequency follows a long-tail distribution. While it is important to recommend popular items that are favored by many users, they are often already recognized by users and are less likely to offer new experiences. In contrast, recently added or less popular items that align with individual preferences can provide novelty and serendipity ( Vargas and Castells 2011 ; Ge et al. 2010 ) . Therefore, recommending both popular and less popular items is important, and such diversity has been shown to improve user satisfaction and service profit ( Yin et al. 2012 ; Liu and Zheng 2020 ) . However, achieving such diversity is challenging, as real-world recommendation data contains missing values that amplify popularity bias. In recommendation data, a zero value indicating no interactions indicates not only disinterest but also missingness due to unobserved items. The missingness mechanism is Missing Not At Random (MNAR) ( Yang et al. 2018 ; Schnabel et al. 2016 ) , and it tends to occur more often in less popular items that are shown to fewer users than in widely displayed popular items. If recommendation models do not address the bias caused by missing data, recommendations become increasingly skewed toward popular items, concentrating user feedback on those items and promoting the lack of diversity ( Mansoury et al. 2020 ; Zheng et al. 2021 ) . Therefore, reducing the bias caused by missing data is an important challenge in recommender systems. This issue can be addressed by collecting complete, unbiased data or by utilizing auxiliary information such as user profiles and contextual information ( Chen et al. 2023 ; Klimashevskaia et al. 2024 ) . However, in real-world settings, collecting such complete data is costly, and auxiliary information is often unavailable or insufficient. Inverse Propensity Scoring (IPS) ( Schnabel et al. 2016 ; Steck 2011 ; Saito et al. 2020 ; Saito 2020 ; Yang et al. 2018 ; Lee et al. 2021 ) is often employed as a simple yet effective approach to reduce this bias, even under such constraints. It estimates the probability that feedback on each item is correctly observed and uses its inverse as a weight to reduce the bias in the observed data. In implicit feedback recommendation, propensity scores based on a power-law function of item frequency have been widely used under the simple assumption that observation probability is proportional to item popularity. However, since the inverse of the power-law function decreases monotonically on a logarithmic scale, applying it as weights tends to excessively suppress popular items. Promoting diversity is important, but popular items are generally of high quality or reflect current trends, and degrading their recommendation accuracy may lead to the loss of valuable recommendation opportunities ( Zhang et al. 2021 ; Klimashevskaia et al. 2024 ; Zhao et al. 2023 ) . To address this issue, we redefine the propensity scores to appropriately reduce popularity bias while avoiding excessive penalization of popular items. Furthermore, we apply the redefined propensity scores to linear autoencoder models ( Steck 2019b ; Steck 2020 ; Moon et al. 2023 ) . Models such as matrix factorization ( Hu et al. 2008 ; Rendle et al. 2022 ) and VAE-based models ( Liang et al. 2018 ; Shenbin et al. 2020 ) typically represent users and items in low-dimensional spaces, which limits their expressive power and reduces item coverage in recommendations ( Ohsaka and Togashi 2023 ) . In contrast, high-dimensional linear autoencoder models ( Steck 2019b ) are capable of representing a wider range of items, yet their recommendations remain biased toward popular items, failing to fully leverage their expressive potential. Weighting based on the redefined propensity scores addresses this problem and achieves a substantial improvement in item coverage while maintaining or improving recommendation accuracy. 1.1. Notation and Preliminaries Vectors are represented by boldface lowercase letters, e.g., a . The i -th element of a vector a is represented by a i . \textbf{0}_{D} and \textbf{1}_{D} represent D -dimensional all-zero and all-one vectors, respectively. Matrices are represented by boldface capital letters, e.g., A . The i -th row of a matrix A is represented by A i∗ , and the j -th column of A is represented by A ∗j . The element (i,j) of a matrix A is denoted by \textbf{A}_{ij} . \textbf{A}^{\top} and \textbf{A}^{-1} denote the transpose and inverse of a matrix A , respectively. \textbf{I}_{D} denotes the D -dimensional identity matrix. \mathop{\text{diag}}(\textbf{A}) is the diagonal of a square matrix A . \mathop{\text{diagMat}}(\textbf{a}) denotes the diagonal matrix whose diagonal is the vector a . Let U be the set of users and I the set of items. Recommendation based on implicit feedback utilizes a binary user-item interaction matrix \textbf{X}\in\{0,1\}^{|U|\times|I|} . Here, \textbf{X}_{ui}=1 indicates that user u has interacted with item i , and \textbf{X}_{ui}=0 otherwise. Item-based collaborative filtering learns a similarity matrix \textbf{B}\in\mathbb{R}^{|I|\times|I|} that captures the similarity between items, where \textbf{B}_{ij} denotes the similarity between item i and item j . The relevance score \textbf{S}_{ui} of

Read the complete text