Exploration on Demand: From Algorithmic Control to User Empowerment
Edoardo Bianchi · 2025
Read Exploration on Demand: From Algorithmic Control to User Empowerment by Edoardo Bianchi (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-21884.txt.
Introduction
Edoardo Bianchi hands the exploration dial to the user. Recommender systems over-specialize: they feed you more of what you already liked until the catalog shrinks to a mirror, the filter bubble. His framework clusters movies by meaning, using sentence-transformer embeddings and an online algorithm that adjusts its own thresholds as items arrive.
On top of the clustering sits an exploration control that samples deliberately from the clusters a user has neglected, widening horizons while openly showing the trade-off between relevance and diversity. MovieLens experiments demonstrate the system working in practice. The philosophical shift is the contribution: diversity as a user right with a visible price, not a knob hidden in the algorithm's basement.
Catalog record
| CREATOR | Edoardo Bianchi |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 30 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2507-21884
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Exploration on Demand: From Algorithmic Control to User Empowerment Edoardo Bianchi Abstract Recommender systems often struggle with over-specialization, which severely limits users’ exposure to diverse content and creates filter bubbles that reduce serendipitous discovery. To address this fundamental limitation, this paper introduces an adaptive clustering framework with user-controlled exploration that effectively balances personalization and diversity in movie recommendations. Our approach leverages sentence-transformer embeddings to group items into semantically coherent clusters through an online algorithm with dynamic thresholding, thereby creating a structured representation of the content space. Building upon this clustering foundation, we propose a novel exploration mechanism that empowers users to control recommendation diversity by strategically sampling from less-engaged clusters, thus expanding their content horizons while preserving relevance. Experiments on the MovieLens dataset demonstrate the system’s effectiveness, showing that exploration significantly reduces intra-list similarity from 0.34 to 0.26 while simultaneously increasing unexpectedness to 0.73. Furthermore, our Large Language Model-based A/B testing methodology, conducted with 300 simulated users, reveals that 72.7% of long-term users prefer exploratory recommendations over purely exploitative ones, providing strong evidence for the system’s ability to promote meaningful content discovery without sacrificing user satisfaction. [orcid=0000-0002-0963-9543, [email protected], ] keywords Recommender Systems ,Adaptive Clustering ,LLM-based Evaluation † † copyrightyear: 2022 † † copyright: Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). † † venue: Beyond Algorithms: Reclaiming the Interdisciplinary Roots of Recommender Systems Workshop @ ACM RecSys 2025, 26 September 2025, Prague, Czech Republic. † † address: Faculty of Engineering, Free University of Bozen-Bolzano, Bozen-Bolzano, 39100, Italy 1 Introduction Recommender systems have become integral to digital content platforms, helping users navigate vast catalogs of movies, music, books, and other media. However, traditional recommendation approaches face persistent challenges in balancing personalization with diversity, often leading to over-specialization that limits users’ exposure to novel content and creates ”filter bubbles” that reinforce existing preferences rather than expanding user horizons. Collaborative filtering methods, while effective at leveraging collective user behavior, rely heavily on historical interaction data, making them ineffective for new users and prone to popularity bias. Content-based approaches often overemphasize similarity metrics, leading to redundant recommendations that fail to introduce meaningful diversity. Both approaches struggle with the exploration-exploitation trade-off: how to balance familiar, relevant content with novel discoveries that might expand user interests. The over-specialization problem is particularly pronounced in dynamic content environments where new items are constantly added and user preferences evolve. Traditional recommender systems optimize for immediate user satisfaction, measured through accuracy metrics like precision and recall, which inherently favor recommendations that match past behavior. This optimization strategy, while improving short-term engagement, can lead to recommendation monotony and reduced long-term user satisfaction. This paper addresses these fundamental limitations through an adaptive recommendation framework that combines semantic clustering with user-controlled exploration. Our approach organizes content using sentence-transformer embeddings and an online clustering algorithm that adapts to content distribution shifts and maintains semantic coherence. The key innovation is an exploration mechanism that allows users to dynamically control the diversity-relevance trade-off in their recommendations, enabling personalized discovery experiences. To evaluate the system’s effectiveness beyond traditional accuracy metrics, we employ a novel Large Language Model (LLM)-based user simulation approach that enables scalable A/B testing of recommendation strategies. This methodology addresses a critical limitation of existing evaluation frameworks, which primarily measure how well recommendations match past interactions while failing to capture the value of content discovery and preference expansion. This work presents the following key contributions: • An adaptive online clustering algorithm that groups items using semantic embeddings with dynamic similarity thresholds; • A user-controlled exploration mechanism that enhances diversity by sampling under-explored clusters while preserving personalization; • A scalable LLM-based user simulation framework for A/B testing recommendation strategies beyond accuracy metrics; • Experimental validation showing improved diversity and unexpectedness while maintaining user satisfaction, especially among long-term users; The remainder of this paper is structured as follows. Section 2 reviews related work on content-based recommendation, clustering approaches, and diversity enhancement techniques. Section 3 details the proposed system architecture, covering the adaptive clustering algorithm, recommendation strategies, and exploration mechanism. Section 4 describes the experimental setup, including dataset preparation, evaluation metrics, and LLM-based user simulation. Section 5 presents comprehensive results and analysis. Section 6 discusses computational complexity and scalability considerations. Section 7 presents limitations of this work, and Section 8 concludes the paper. 2 Related Work 2.1 Content-Based Recommendation Systems Content-based filtering has evolved from metadata-based approaches to sophisticated systems leveraging rich item representations from unstructured content Lops P. 2019 . Recent deep learning advances enable more effective content representations, with studies showing that visual features from CNNs Filho et al. 2017 and NLP techniques like word embeddings Berbatova 2019 significantly enhance semantic content profiling beyond traditional metadata approaches. 2.2 Clustering-Based Approaches Clustering techniques have proven effective for organizing content and enhancing recommendation diversity. Zhang and Hurley Zhang and Hurley 2009 demonstrated that partitioning user profiles into distinct taste segments improves recommendation novelty and balance. Theoretical work by Cohen-Addad et al. Cohen-Addad et al. 2019 provides foundations for online k-means clustering with performance guarantees, while Choromanska and Monteleoni Choromanska and Monteleoni 2012 emphasized adaptive mechanisms for streaming data that minimize concept drift while preserving cluster coherence. 2.3 Diversity and Exploration in Recommendations The importance of recommendation diversity is well-established, with Ziegler et al. Ziegler et al. 2005 showing that topic diversification improves user satisfaction despite potential accuracy trade-offs. Adamopoulos and Tuzhilin Adamopoulos and Tuzhilin 2014 formalized unexpectedness as distinct from novelty and serendipity, providing frameworks for evaluation beyond accuracy metrics. However, most existing exploration approaches lack user control and transparency, which our work addresses through semantic clustering. 2.4 Evaluation Beyond Accuracy Traditional evaluation focuses on accuracy metrics like precision and recall, which are biased toward reinforcing existing preferences Aggarwal 2016 ; Francesco Ricci 2015 . Recent research explores alternative methodologies, including the use of LLMs for simulating human behavior in recommendation scenarios, providing reliable preference judgments that correlate well with human evaluators Bougie and Watanabe 2025 ; Zhang et al. 2024 . These approaches build on broader research demonstrating that LLMs can effectively approximate human behavioral patterns and decision-making processes across various domains Xie et al. 2024 ; Li et al. 2025 . 3 Methodology 3.1 System Architecture Overview The proposed recommendation system addresses three fundamental challenges in content recommendation: information overload, cold-start scenarios, and limited content diversity. The system employs a modular architecture consisting of five interconnected components designed to work together seamlessly. The Item Embedding Module leverages state-of-the-art sentence transformer models to convert textual metadata into dense semantic representations. The Adaptive Clustering Module groups items into semantically coherent clusters that evolve with the content catalog. The Cold-Start Recommendation Module enables immediate personalization for new users through keyword-based preference specification. The Personalized Recommendation Module generates tailored suggestions for returning users by analyzing interaction history. Finally, the Exploration Control Module allows users to dynamically adjust the diversity-relevance balance in their recommendations. 3.2 Item and User Representation 3.2.1 Item Space Definition Each item in the system is characterized by comprehensive metadata: \displaystyle\text{Item}=\{ID,Title,Tags,Keywords,Description\} (1) where Tags consist of single descriptive words, Keywords include multi-word phrases, and Description provides a textual summary of content. This rich representation enables nuanced semantic understanding that goes beyond simple keyword matching. 3.2.2 Semantic Embeddings