With Friends Like These, Who Needs Explanations? Evaluating User Understanding of Group Recommendations Conference: 33rd ACM Conference on User Modeling, Adaptation and Personalization; June 16–19, 2025; New York City, NY, USA 33rd ACM Conference on User Modeling, Adaptation and Personalization (UMA
Cedric Waterschoot, Raciel Yera Toledo, Nava Tintarev, Francesco Barile · 2025
Read With Friends Like These, Who Needs Explanations? Evaluating User Understanding of Group Recommendations Conference: 33rd ACM Conference on User Modeling, Adaptation and Personalization; June 16–19, 2025; New York City, NY, USA 33rd ACM Conference on User Modeling, Adaptation and Personalization (UMA by Cedric Waterschoot, Raciel Yera Toledo, Nava Tintarev, Francesco Barile (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-04273.txt.
Introduction
Cedric Waterschoot and three co-authors test in 2025 whether explanations help people understand group recommendations. Recommender systems often merge a group's tastes by social-choice rules, which can leave a minority member poorly served. Explanations are the usual fix, yet their effect on real understanding was unmeasured.
In a trial with 271 people, the team varied both the merging rule and the explanation style: none, text, or text plus visuals. Understanding was checked by self-report and by tasks like predicting the system's output and spotting its errors. Minority scenarios made the test deliberately hard. The study separates feeling informed from actually grasping the system, a distinction designers of group tools should not blur.
Catalog record
| CREATOR | Cedric Waterschoot, Raciel Yera Toledo, Nava Tintarev, Francesco Barile |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 47 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-04273
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
With Friends Like These, Who Needs Explanations? Evaluating User Understanding of Group Recommendations Conference: 33rd ACM Conference on User Modeling, Adaptation and Personalization; June 16–19, 2025; New York City, NY, USA 33rd ACM Conference on User Modeling, Adaptation and Personalization (UMAP ’25), June 16–19, 2025, New York City, NY, USA DOI: 10.1145/3699682.3728345 ISBN: 979-8-4007-1313-2/2025/06 CCS: Human-centered computing User studies CCS: Human-centered computing Social recommendation CCS: Information systems Recommender systems Cedric Waterschoot, Raciel Yera Toledo, Nava Tintarev, Francesco Barile Abstract . Group Recommender Systems (GRS) employing social choice-based aggregation strategies have previously been explored in terms of perceived consensus, fairness, and satisfaction. At the same time, the impact of textual explanations has been examined, but the results suggest a low effectiveness of these explanations. However, user understanding remains fairly unexplored, even if it can contribute positively to transparent GRS. This is particularly interesting to study in more complex or potentially unfair scenarios when user preferences diverge, such as in a minority scenario (where group members have similar preferences, except for a single member in a minority position). In this paper, we analyzed the impact of different types of explanations on user understanding of group recommendations. We present a randomized controlled trial ( n = 271 n=271 ) using two between-subject factors: (i) the aggregation strategy (additive, least misery, and approval voting), and (ii) the modality of explanation (no explanation, textual explanation, or multimodal explanation). We measured both subjective (self-perceived by the user) and objective understanding (performance on model simulation, counterfactuals and error detection). In line with recent findings on explanations for machine learning models, our results indicate that more detailed explanations, whether textual or multimodal, did not increase subjective or objective understanding. However, we did find a significant effect of aggregation strategies on both subjective and objective understanding. These results imply that when constructing GRS, practitioners need to consider that the choice of aggregation strategy can influence the understanding of users. Post-hoc analysis also suggests that there is value in analyzing performance on different tasks, rather than through a single aggregated metric of understanding. 1. Introduction Group Recommender Systems (GRS) process the preferences of multiple individuals to derive a single recommendation tailored to suit the group as a whole. To achieve this, previous research has introduced social choice-based aggregation strategies ( Masthoff 2015 ) . These strategies present a range of options in which the preferences of individual group members are aggregated and present an opportunity to adapt GRS to different group configurations and needs. However, the implementation of these strategies on top of already complex recommender systems may result in non-transparent models, hindering user understanding. This is particularly interesting in scenarios in which user preferences diverge, leading to potentially unfair scenarios. Social choice-based explanations for group recommendations have been used as a way to intuitively explain the process behind group recommendations ( Najafian and Tintarev 2018 ; Kapcak et al. 2018 ) . These textual explanations have been evaluated in terms of users’ fairness perception, consensus perception, and satisfaction in user studies with contrasting results ( Tran et al. 2019 ; Barile et al. 2023 ) . Hence, their effectiveness in improving user understanding of the strategies used to generate recommendations for the group still remains to be seen. In this study, we build upon these works, focusing on the understandability of explanations and in turn, group recommendations themselves. We formulate the following research questions: RQ1. Do explanations increase users’ understanding of the underlying social choice-based aggregation strategies? RQ2. Does the effect of explanations on the understandability of GRS vary depending on the underlying social-based aggregation strategy? Inspired by Wang and Yin 2021 , we constructed a randomized controlled trial with two between-subject factors ( 3x3 design): (i) the social choice-based aggregation strategy used for generating group recommendations, and (ii) the type of explanation provided to the participant. We included three explanation types: no explanation (control group), textual explanation only and, a multimodal explanation combining text with a visualization. To measure the participant’s understanding of the strategy and GRS, we made use of objective and subjective understanding as described in previous research ( Rong et al. 2024 ; Knijnenburg et al. 2011 ; Radensky et al. 2022 ) . We present minority scenarios, i.e. group scenarios in which user preferences are similar except for one single member, a complex and potentially unfair scenario. Our paper makes the following contributions: • We conduct a preregistered, randomized controlled trail ( n=271 ) to analyze the effect of different explanation types on user understanding. • We show that more detailed explanations do not positively or negatively impact user understanding. • We outline that the choice of aggregation strategy can impact understandability down the line. • We find value in measuring understandability using a variety of tasks as opposed to a single metric. The remainder of the paper is structured as follows. First, we outline the literature on GRS and understandability of Recommender Systems. Based on this body of previous work, we formulate our hypotheses. Second, we present our materials, including the measurement of understandability, explanation types, and experimental set-up. Subsequently, we describe the results of our experiment followed by a discussion of the results and their implications. 2. Background In this section, we introduce the literature on group recommendation and social choice-based aggregation strategies. Additionally, we outline recent work combining understandability and Explainable AI (XAI) with Recommender Systems and highlight the research gap related to the understandability of GRS in particular. 2.1. Group Recommendation Table 1. Social choice-based aggregation strategies derived from ( Tran et al. 2019 ; Barile et al. 2023 ; Felfernig et al. 2018 ) Strategy Type Procedure Additive Utilitarion (ADD) Consensus Recommends the item with the highest sum of all group members’ ratings Fairness (FAI) Consensus Ranking and recommending items according to how individuals choose them in turn Approval Voting (APP) Majority Recommends the item with the highest number of ratings above a predefined threshold Least Misery (LMS) Borderline Recommends the item which has the highest of all lowest per-item ratings Most Pleasure (MPL) Borderline Recommends the item with the highest individual group member rating Recommendation systems tailored to groups, rather than the preferences of a single user, are rising in demand due to applications in fields such as tourism ( Chen et al. 2021 ) and music ( Najafian and Tintarev 2018 ) . These Group Recommender Systems (GRS) are designed to support the decision-making process of multiple people simultaneously ( Masthoff and Delić 2022 ) . The preferences of the individual group members need to be processed in a way that the recommendation reflects the group as a whole. Based on Social Choice Theory ( Kelly 2013 ) , social choice-based aggregation strategies are employed for aggregating the preferences of the group members to obtain a recommendation for the group as a whole ( Masthoff 2015 ) . Examples of often used social-choice based approaches are summarized in Table 1 . A range of different strategies have been introduced in the literature and can be categorized as follows: consensus-based , borderline and, majority-based . The latter makes use of only the most popular items or ratings ( Senot et al. 2010 ) . An example of such