Rank, Don’t Generate: Statement-level Ranking for Explainable Recommendation DOI: XXXXXXX.XXXXXXX Conference: Make sure to enter the correct conference title from your rights confirmation email; June 03–05, 2018; Woodstock, NY ISBN: 978-1-4503-XXXX-X/2018/06 CCS: Information systems Learning to rank CCS: Information systems Top-k retrieval in databases CCS: Information systems Language models CCS: Information systems Clustering CCS: Information systems Recommender systems CCS: Information systems Sentiment analysis CCS: Information systems Clustering and classification

Ben Kabongo, Arthur Satouf, Vincent Guigue · 2026

Read Rank, Don’t Generate: Statement-level Ranking for Explainable Recommendation DOI: XXXXXXX.XXXXXXX Conference: Make sure to enter the correct conference title from your rights confirmation email; June 03–05, 2018; Woodstock, NY ISBN: 978-1-4503-XXXX-X/2018/06 CCS: Information systems Learning to rank CCS: Information systems Top-k retrieval in databases CCS: Information systems Language models CCS: Information systems Clustering CCS: Information systems Recommender systems CCS: Information systems Sentiment analysis CCS: Information systems Clustering and classification by Ben Kabongo, Arthur Satouf, Vincent Guigue (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-03724.txt.

Introduction

Ben Kabongo and colleagues argued that a system should pick its reasons for a suggestion from a ranked list, rather than writing them up from scratch. Their StaR approach ranks short candidate lines drawn from user reviews and returns the best ones as the reason. That curbs made-up claims and lets each claim be checked on its own. Each line must say something real, cover one point only, and not repeat another line. Simple popularity baselines proved surprisingly strong when ranked per item.

Catalog record

CREATORBen Kabongo, Arthur Satouf, Vincent Guigue
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE57213
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.03724

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Rank, Don’t Generate: Statement-level Ranking for Explainable Recommendation DOI: XXXXXXX.XXXXXXX Conference: Make sure to enter the correct conference title from your rights confirmation email; June 03–05, 2018; Woodstock, NY ISBN: 978-1-4503-XXXX-X/2018/06 CCS: Information systems Learning to rank CCS: Information systems Top-k retrieval in databases CCS: Information systems Language models CCS: Information systems Clustering CCS: Information systems Recommender systems CCS: Information systems Sentiment analysis CCS: Information systems Clustering and classification Ben Kabongo, Arthur Satouf, Vincent Guigue Abstract . Textual explanations, generated with large language models (LLMs), are increasingly used to justify recommendations. Yet, evaluating these explanations remains a critical challenge. Lexical n n -gram metrics struggle with paraphrases; semantic metrics reward similarity without ensuring factual grounding; LLM-based evaluators are highly prompt-dependent, often proprietary, and hard to replicate, while current evaluation protocols provide little insight into which explanatory factors are faithful and relevant. Meanwhile, proposed models tend to hallucinate unsupported claims or default to generic rationales. We advocate a shift in objective: rank, don’t generate . We formalize explainable recommendation as a statement-level ranking problem, where systems rank candidate explanatory statements derived from reviews and return the top- k k as explanation. This formulation mitigates hallucination by construction and enables fine-grained factual analysis. It also models factor importance through relevance scores and supports standardized, reproducible evaluation with established ranking metrics. Meaningful assessment, however, requires each statement to be explanatory (item facts affecting user experience), atomic (one opinion about one aspect), and unique (paraphrases consolidated), which is challenging to obtain from noisy reviews. We address this with (i) an LLM-based extraction pipeline producing explanatory and atomic statements, and (ii) a scalable, semantic clustering method consolidating paraphrases to enforce uniqueness. Building on this pipeline, we introduce StaR , a benchmark for statement ranking in explainable recommendation, constructed from four Amazon Reviews 2014 product categories. We evaluate popularity-based baselines and state-of-the-art models under global-level (all statements) and item-level (target item statements) ranking. Popularity baselines are competitive in global-level ranking but outperform state-of-the-art models on average in item-level ranking, exposing critical limitations in personalized explanation ranking. Code, data, and supplementary materials are released to support reproducible research and future work. 1 1 1 https://github.com/BenKabongo25/Statement_Ranking_Explainable_Recommendation Better explanation evaluation needs better units: rank statements, not paragraphs. Figure 1. Better explanation evaluation needs better units: rank statements, not paragraphs. Better explanation evaluation needs better units: rank statements, not paragraphs. 1. Introduction Recommender systems play a central role in helping users navigate large catalogs of content, products, or services by delivering personalized suggestions. However, the growing reliance on deep learning architectures and techniques ( 10 ; 11 ; 26 ; 32 ) has made these systems increasingly opaque and difficult to explain or interpret. Explainable recommendation addresses this issue by providing human-understandable justifications for why an item is recommended, which is crucial for building user trust and improving system transparency. Among existing paradigms, textual explanations are particularly appealing. Recent methods ( 13 ; 19 ; 21 ; 23 ; 27 ; 42 ) increasingly leverage large language models (LLMs) ( 8 ; 37 ; 44 ) to generate natural-language rationales. However, reliably assessing the quality of generated explanations remains a critical challenge. Early evaluation protocols relied on lexical n -gram overlap metrics ( 4 ; 24 ; 30 ) , which struggle with synonymy and paraphrasing. Learned semantic metrics ( 33 ; 45 ; 46 ) reduce dependence on surface form, but semantic similarity does not guarantee factual grounding in the underlying evidence ( 12 ; 14 ; 15 ) . LLM-based evaluators ( 9 ; 25 ; 16 ) can provide richer judgments, but are highly prompt-dependent and often proprietary, making them hard to reproduce ( 34 ; 43 ) . Beyond evaluation, generation itself introduces additional failure modes: explainable models may hallucinate unsupported claims or default to generic rationales rather than interaction-specific justifications ( 14 ; 18 ; 42 ) . Moreover, explanations are intrinsically multi-faceted: multiple factors may jointly justify a recommendation and vary in importance for a given user (e.g., cast versus storyline for movies), whereas current protocols typically score whole paragraphs holistically, offering limited insight into which factors are supported and failing to reflect their relative importance. Statement-level ranking for explainable recommendation. These limitations motivate reframing text-based explainable recommendation as a ranking problem over explanatory passages, which we call statements . Following prior work ( 18 ; 20 ; 41 ) , we advocate a paradigm shift: rank, don’t generate . Instead of producing a free-form paragraph, the system ranks candidate statements for each user–item interaction, from the most to the least relevant, and returns the top- k as the explanation. This formulation offers three key advantages. First, it decomposes explanations into verifiable units grounded in user reviews, mitigating hallucination and enabling fine-grained faithfulness analysis. Second, ranking captures relative factor importance via relevance scores, allowing the system to prioritize the most important aspects for a given interaction. Third, it enables standardized, reproducible evaluation with established ranking metrics (e.g., Precision, Recall, and NDCG), supporting objective comparisons across methods. Despite its appeal, statement-level ranking faces fundamental challenges in dataset construction and evaluation. In particular, meaningful assessment requires each statement to satisfy three properties: (i) explanatoriness , describing general item facts that may affect user experience rather than circumstantial or personal information with no explanatory value; (ii) atomicity , expressing a single opinion about a single item aspect; and (iii) uniqueness , ensuring that (multiple) semantically equivalent declarations are grouped into a single target to avoid skewing metrics. Existing datasets typically consist of free-form reviews that interleave explanatory content with noise, merge multiple aspects, and vary widely in style, making it difficult to extract statement sets that meet these requirements. To address these challenges, prior work introduces Extra ( 18 ) , which extracts frequent review phrases as statements and deduplicates paraphrases using lexical n -gram similarity. In practice, purely lexical clustering often fails to consolidate semantically equivalent paraphrases expressed with different wording, and extraction from raw reviews remains challenged by non-explanatory and non-atomic content. Satisfying the above statement properties therefore requires robust extraction to filter noise while producing explanatory, atomic units, as well as semantics-aware clustering to group paraphrases beyond surface-level similarity. Our work. We propose improved methods for statement extraction and paraphrase clustering. For extraction, we introduce a two-stage LLM pipeline: (i) candidate extraction , which produces explanatory, atomic statements while filtering review noise, and (ii) verification , which removes non-compliant outputs and semantic duplicates. For clustering, we propose a scalable, semantics-driven procedure: (i) approximate nearest-neighbor search retrieves semantically similar candidates using dense embeddings, (ii) pairwise filtering removes false positives with a cross-encoder, and (iii) refinement builds a similarity graph from validated pairs, extracts connected components as initial clusters, and further splits low-cohesion components to enforce intra-cluster consistency. Building on this pipeline, we introduce StaR ( Sta tement R anking), a new benchmark for statement ranking in explainable recommendation, constructed from four categories ( Toys , Clothing , Beauty , and Sports ) of the Amazon Reviews 2014 dataset ( 28 ) . This yields augmented datasets tailored to statement ranking, with 115K–294K interactions and 718K–1.3M user–item–statement triplets after clustering. We validate StaR by measuring the required statement properties: human annotation on sampled examples and LLM-based

Read the complete text