The Pitfalls of Growing Group Complexity: LLMs and Social Choice-Based Aggregation for Group Recommendations

Cedric Waterschoot, Nava Tintarev, Francesco Barile · 2025

Read The Pitfalls of Growing Group Complexity: LLMs and Social Choice-Based Aggregation for Group Recommendations by Cedric Waterschoot, Nava Tintarev, Francesco Barile (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-05016.txt.

Introduction

Cedric Waterschoot, Nava Tintarev, and Francesco Barile probe when language models can merge a group's tastes into one recommendation. Their paper appeared in 2025. Group recommenders traditionally use social choice rules to combine ratings. The team asked whether large language models can apply those rules from plain prompts, and how accuracy bends as groups grow. They varied group size, item counts, the model used, prompt wording, and whether examples or explanations were included.

Accuracy held for small groups, then fell once scenarios passed about one hundred ratings. Models differed in how steeply they degraded. Showing examples in the prompt helped a lot at high complexity; domain hints and asking for explanations did not. The authors argue group complexity belongs in every future evaluation of such systems, since real groups are rarely small and tidy.

Catalog record

CREATORCedric Waterschoot, Nava Tintarev, Francesco Barile
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE42 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-05016

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

The Pitfalls of Growing Group Complexity: LLMs and Social Choice-Based Aggregation for Group Recommendations Cedric Waterschoot, Nava Tintarev, Francesco Barile Abstract . Large Language Models (LLMs) are increasingly applied in recommender systems aimed at both individuals and groups. Previously, Group Recommender Systems (GRS) often used social choice-based aggregation strategies to derive a single recommendation based on the preferences of multiple people. In this paper, we investigate under which conditions language models can perform these strategies correctly based on zero-shot learning and analyse whether the formatting of the group scenario in the prompt affects accuracy. We specifically focused on the impact of group complexity (number of users and items), different LLMs, different prompting conditions, including In-Context learning or generating explanations, and the formatting of group preferences. Our results show that performance starts to deteriorate when considering more than 100 100 ratings. However, not all language models were equally sensitive to growing group complexity. Additionally, we showed that In-Context Learning (ICL) can significantly increase the performance at higher degrees of group complexity, while adding other prompt modifications, specifying domain cues or prompting for explanations, did not impact accuracy. We conclude that future research should include group complexity as a factor in GRS evaluation due to its effect on LLM performance. Furthermore, we showed that formatting the group scenarios differently, such as rating lists per user or per item, affected accuracy. All in all, our study implies that smaller LLMs are capable of generating group recommendations under the right conditions, making the case for using smaller models that require less computing power and costs. Conference: Adjunct Proceedings of the 33rd ACM Conference on User Modeling, Adaptation and Personalization; June 16–19, 2025; New York City, NY, USA Adjunct Proceedings of the 33rd ACM Conference on User Modeling, Adaptation and Personalization (UMAP Adjunct ’25), June 16–19, 2025, New York City, NY, USA DOI: 10.1145/3708319.3733659 ISBN: 979-8-4007-1399-6/2025/06 CCS: Information systems Recommender systems CCS: Computing methodologies Natural language generation † † cc-license: by 1. Introduction Group Recommender Systems (GRS) extend traditional recommender systems to obtain a single recommendation required to suit the preferences of a group of people. To process the potentially diverging ratings of multiple people, prior research has introduced social choice-based aggregation strategies ( Masthoff, 2015 ) . Recently, Large Language Models (LLMs) have been used to obtain group recommendations by providing them the individual ratings of group members ( Tommasel, 2024 ; Feng et al., 2025a ) . Social choice-based aggregation strategies were used as comparison for the LLM-generated recommendations. However, LLMs were not explicitly prompted to perform these strategies, even though strategies provide opportunities to adapt GRS to specific conditions and factors such as fairness and satisfaction ( Barile et al., 2023 ; Tran et al., 2019 ) . Understanding how the applied strategies align with, or differ from, LLM strategies helps us understand LLM capabilities for GRS better. While recommender systems typically deal with large amounts of data, the number of total ratings the model has to consider, remains fairly overlooked. This complexity is important to consider since adding additional information to the prompt influences the output of LLMs ( Zamfirescu-Pereira et al., 2023 ) . While the accuracy of traditional aggregation techniques for GRS are not affected by a growing number of items or group members, it is unclear how robust LLMs are against increasing group complexity. It is also unclear under which conditions this kind of robustness is present. Therefore, we formulate the following research questions: RQ1. Are LLMs robust against increasing group complexity, defined as the total number of ratings to consider, i.e. the product of group members and items, when applying aggregation strategies? RQ2. Do additional prompt conditions; e.g., generating explanations, adding examples, or adding domain cues; impact the accuracy of LLMs when applying aggregation strategies? RQ3. Does the format of the group; e.g., ratings per user or per item; impact the accuracy of LLMs when applying aggregation strategies? Table 1. Social choice-based aggregation strategies derived from ( Tran et al., 2019 ; Barile et al., 2023 ; Felfernig et al., 2018 ; Waterschoot et al., 2025 ) Strategy Type Procedure Additive Utilitarian (ADD) Consensus Recommends the item with the highest sum of all group members’ ratings Approval Voting (APP) Majority Recommends the item with the highest number of ratings above a predefined threshold Least Misery (LMS) Borderline Recommends the item which has the highest of all lowest per-item ratings Most Pleasure (MPL) Borderline Recommends the item with the highest individual group member rating To investigate these questions, we created 1,000 fictitious groups with randomly 2,4 , or 8 members and 5,10,25 , or 50 potential items to recommend. For each group, multiple LLMs were prompted to derive a group recommendation using a randomly chosen social choice-based aggregation strategy, presented to the LLM using a social choice-based explanation introduced in previous work ( Najafian and Tintarev, 2018 ; Kapcak et al., 2018 ) . Furthermore, we explored several additional conditions that can affect LLM performance in generating correct recommendations at higher degrees of group complexity. We investigated whether prompting for natural language explanations affected the accuracy, a widespread use case of language models. Additionally, we tested whether adding domain cues (movie titles instead of anonymous items) or additional examples (in-context learning) improved the accuracy of recommendations generated by LLMs. All in all, our paper makes the following contributions: • We highlight the importance of using a selection of LLMs due to uneven performance when generating group recommendations. • We show that group complexity is an important factor in the evaluation of group recommendations generated by LLMs due to its effect on performance. • We find that the prompting strategy of In-Context Learning significantly improves the capability of LLMs to apply social choice-based aggregation strategies at higher group complexity correctly. • We show that the formatting of the group scenario impacts the accuracy of LLMs when applying aggregation strategies. 2. Background In this section, we introduce the literature on Group Recommendation Systems (GRS) and social choice-based aggregation strategies. Additionally, we outline previous work on using LLMs for (group) recommendation and highlight the need for a more robust evaluation of LLM-generated recommendations in the context of standard aggregation methods for GRS. 2.1. Group Recommendation While traditional recommender systems present output for a single user, GRS need to simultaneously process the preferences of multiple group members ( Masthoff and Delić, 2022 ) . These systems are in rising demand and are being applied in various fields such as tourism ( Chen et al., 2021 ) , music ( Najafian and Tintarev, 2018 ) and restaurant recommendation ( Barile et al., 2021 ) . To derive a single recommendation which reflects the ratings of individual group members, individual ratings need to be aggregated. Inspired by Social Choice Theory ( Kelly, 2013 ) , social choice-based aggregation strategies have been proposed to present a range of distinct options to process individual ratings into a single outcome ( Masthoff, 2015 ; Masthoff, 2004 ) . Even when more advanced approaches are proposed in which social factors or other dynamics influencing group decision-making are incorporated (see e.g. ( Nguyen et al., 2019 ; Delic et al., 2018 ) ), these strategies are still widely used

Read the complete text