Are We Solving a Well-Defined Problem? A Task-Centric Perspective on Recommendation Tasks DOI: XXXXXXX.XXXXXXX Conference: Draft; March 2025; ACM Template ISBN: 978-1-4503-XXXX-X/18/06 Note: This work extends the main idea presented in ( Sun 2024 ) by providing a clearer formulation of RecSys tasks

Aixin Sun · 2025

Read Are We Solving a Well-Defined Problem? A Task-Centric Perspective on Recommendation Tasks DOI: XXXXXXX.XXXXXXX Conference: Draft; March 2025; ACM Template ISBN: 978-1-4503-XXXX-X/18/06 Note: This work extends the main idea presented in ( Sun 2024 ) by providing a clearer formulation of RecSys tasks by Aixin Sun (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-21188.txt.

Introduction

Aixin Sun asks whether recommender systems research is solving a well-defined problem. Most studies adopt one broad definition: suggest items users will like, based on past behavior. Yet their models are judged on data from real platforms, which quietly carry the specifics of one domain and task.

Sun dissects how a recommendation task is framed: what goes in and comes out, how time is handled, and how candidate items are chosen. Each choice shapes offline evaluation. Complications such as decision costs, multi-step engagement, and invisible interactions also bend model design. The paper weighs task specificity against model generality, arguing that clear task definitions are the ground for fair tests and sound solutions.

Catalog record

CREATORAixin Sun
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE51 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2503-21188

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Are We Solving a Well-Defined Problem? A Task-Centric Perspective on Recommendation Tasks DOI: XXXXXXX.XXXXXXX Conference: Draft; March 2025; ACM Template ISBN: 978-1-4503-XXXX-X/18/06 Note: This work extends the main idea presented in ( Sun 2024 ) by providing a clearer formulation of RecSys tasks and a more detailed discussion on user-item interactions. CCS: Information systems Recommender systems Aixin Sun Abstract . Recommender systems (RecSys) leverage user interaction history to predict and suggest relevant items, shaping user experiences across various domains. While many studies adopt a general problem definition, i.e., to recommend preferred items to users based on past interactions, such abstraction often lacks the domain-specific nuances necessary for practical deployment. However, models are frequently evaluated using datasets from online recommender platforms, which inherently reflect these specificities. In this paper, we analyze RecSys task formulations, emphasizing key components such as input-output structures, temporal dynamics, and candidate item selection. All these factors directly impact offline evaluation. We further examine the complexities of user-item interactions, including decision-making costs, multi-step engagements, and unobservable interactions, which may influence model design and loss functions. Additionally, we explore the balance between task specificity and model generalizability, highlighting how well-defined task formulations serve as the foundation for robust evaluation and effective solution development. By clarifying task definitions and their implications, this work provides a structured perspective on RecSys research. The goal is to help researchers better navigate the field, particularly in understanding specificities of the RecSys tasks and ensuring fair and meaningful evaluations. 1. Introduction In many recommender systems (RecSys) studies, solutions are proposed for a commonly adopted problem: given a set of users, items, and their interactions, the goal is to recommend items that align with users’ interests or preferences. While this general problem definition captures common patterns across recommendation scenarios, its abstraction overlooks critical details needed for practical RecSys applications. Furthermore, discussions on task definition often lack clarity, particularly regarding its scope and practical implications. At the same time, RecSys research is closely tied to real-world applications, where models are primarily evaluated using datasets obtained from operational recommender platforms. The mismatch between abstract problem definitions and domain-specific evaluations leads to inconsistent settings and findings across experiments ( Zangerle and Bauer 2022 ; McElfresh et al. 2022 ) . In this paper, we present a comprehensive analysis of different RecSys tasks, highlighting their key distinctions and implications for evaluation and real-world deployment. Recommender systems are not a new research area, and numerous influential works exist. We begin our discussion in Section 2 with a review of several highly cited works in RecSys, emphasizing task formulations. Interestingly, key factors in RecSys were well defined and discussed decades ago. Yet, the community still disagrees on the choice of baselines and datasets ( Ivanova et al. 2023 ; Beel et al. 2024 ) . One main contribution of this paper is a detailed examination of task definition in Section 3 , focusing primarily on the input and output of a mapping function, i.e., a recommender. We discuss a missing element in task formulation i.e., time, and the selection of candidate items for recommendation, both of which have a significant impact on offline evaluation. Another contribution of this paper is a detailed analysis of the life cycle of user-item interactions in Section 4 . In the pre-interaction judgment stage, users are presented with a list of choices or recommended items. Users incur costs when making informed (or sometimes uninformed) decisions to select a recommendation, which serves as a clear indication of user preferences. We then explore the complexity of user-item interactions. For example, some interactions can involve multiple steps, as in e-commerce, where a user may add an item to their cart, proceed to payment, await delivery, interact with the product, and possibly return it. In music streaming, users are not offered a choice but can only skip or fastforward a recommended song. In some cases, true user-item interactions may not be directly observable as in job recommendation. Based on the task definition and the framework for understanding it, we present our perspectives in Section 5 . We consider (i) the ultimate goal of recommendation, i.e., reducing user cost, (ii) the relationship between task formulation, solution, and evaluation, and (iii) the intersection of task specificity and model generalizability. In our discussion, we use a few examples to show the differences in RecSys tasks based on the presented dimensions, i.e., candidate item selection, pre-interaction judgment, and user-item interaction. We hope that this paper provides researchers, particularly those new to recommender systems, with a clearer understanding of the research topic. After establishing a clear understanding, subsequent research can precisely define specific RecSys tasks, select appropriate baselines, and design experiments that account for task specificity. 2. A Historical Review of Task Formulation We begin with four of the most widely cited papers, 1 1 1 Citation counts were obtained via a Google Scholar search for papers with the title words “recommender systems” or “recommender system” on February 12, 2025. each with over 8,000 citations. As foundational works in this field, these papers have influenced many researchers, and the tasks they define have likely shaped numerous follow-up studies. The four papers reviewed were published between 2004 and 2011 ( Herlocker et al. 2004 ; Koren et al. 2009 ; Ricci et al. 2011 ; Adomavicius and Tuzhilin 2005 ) . The Recommendation Problem. In one of the most influential survey papers in RecSys, Adomavicius and Tuzhilin 2005 formally define the recommendation problem as follows: Definition 0 (Recommendation Problem). Let U be the set of all users and let I be the set of all possible items that can be recommended. Let r be a utility function that measures the usefulness of item i to user u , i.e., r\in U\times I\rightarrow R , where R is a totally ordered set ( i.e., nonnegative integers or real numbers within a certain range). Then, for each user u\in U , we want to choose such item i^{\prime}\in I that maximizes the user’s utility: (1) \forall u\in U,i^{\prime}_{u}=\arg\max_{i\in I}r(u,i) The authors further note that “the utility of an item is usually represented by a rating, which indicates how a particular user liked a particular item,” a concept commonly known as explicit feedback . Nevertheless, in most subsequent RecSys studies, implicit feedback has become far more prevalent ( Rendle et al. 2009 ) . As a result, the utility of an item to a user is often inferred from binary feedback, whether the user has interacted with the item, such as listening to a song, purchasing a watch, or staying at a hotel. Notably, the difference between explicit and implicit feedback primarily affects data modeling and the loss function design when learning r , while the core recommendation problem remains unchanged. Collaborative Filtering. Koren et al. 2009 do not formally define tasks in recommender systems but instead compare the two primary approaches: content filtering and collaborative filtering. They describe content filtering as a method that creates user or item profiles to capture their characteristics, enabling the recommender to associate users with relevant items. In contrast, collaborative filtering relies exclusively on past user-item interactions, such as previous transactions or product ratings. The authors highlight that “a major appeal of collaborative filtering

Read the complete text