Following the Eye-Tracking Evidence: Established Web-Search Assumptions Fail in Carousel Interfaces Conference: Proceedings of the 49th International ACM SIGIR Conference on Research and Development in Information Retrieval; July 20–24, 2026; Melbourne, VIC, Australia Proceedings of the 49th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR ’26), July 20–24, 2026, Melbourne, VIC, Australia DOI: 10.1145/3805712.3809657 ISBN: 979-8-4007-2599-9/2026/07 CCS: Information systems Recommender systems CCS: Human-centered computing User studies
Jingwei Kang, Maarten de Rijke, Harrie Oosterhuis · 2026
Read Following the Eye-Tracking Evidence: Established Web-Search Assumptions Fail in Carousel Interfaces Conference: Proceedings of the 49th International ACM SIGIR Conference on Research and Development in Information Retrieval; July 20–24, 2026; Melbourne, VIC, Australia Proceedings of the 49th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR ’26), July 20–24, 2026, Melbourne, VIC, Australia DOI: 10.1145/3805712.3809657 ISBN: 979-8-4007-2599-9/2026/07 CCS: Information systems Recommender systems CCS: Human-centered computing User studies by Jingwei Kang, Maarten de Rijke, Harrie Oosterhuis (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-21019.txt.
Introduction
Kang, de Rijke, and Oosterhuis test whether web-search behavior rules hold for carousel interfaces, the row-based layouts standard on streaming services. Past work assumed patterns from single-list search, like F-pattern viewing and the examination hypothesis, would carry over. From an eye-tracking data set, the team finds gaze follows the F-pattern for vertical examination alone, not for horizontal swiping. When a click occurs, gaze follows an L-pattern unique to carousels. The examination hypothesis fails here, and users ignore headings, looking straight at content items.
Catalog record
| CREATOR | Jingwei Kang, Maarten de Rijke, Harrie Oosterhuis |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 47231 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.21019
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Following the Eye-Tracking Evidence: Established Web-Search Assumptions Fail in Carousel Interfaces Conference: Proceedings of the 49th International ACM SIGIR Conference on Research and Development in Information Retrieval; July 20–24, 2026; Melbourne, VIC, Australia Proceedings of the 49th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR ’26), July 20–24, 2026, Melbourne, VIC, Australia DOI: 10.1145/3805712.3809657 ISBN: 979-8-4007-2599-9/2026/07 CCS: Information systems Recommender systems CCS: Human-centered computing User studies Jingwei Kang, Maarten de Rijke, Harrie Oosterhuis Abstract . Carousel interfaces have been the de-facto standard for streaming media services for over a decade. Yet, there has been very little research into user behavior with such interfaces, which thus remains poorly understood. Due to this lack of empirical research, previous work has assumed that behaviors established in single-list web-search interfaces, such as the F-pattern and the examination hypothesis, also apply to carousel interfaces, for instance when designing click models or evaluation metrics. We analyze a recently-released interaction and examination dataset resulting from an eye-tracking study performed on carousel interfaces to verify whether these assumptions actually hold. We find that (i) the F-pattern holds only for vertical examination and not for horizontal swiping; additionally, we discover that, when conditioned on a click, user examination follows an L-pattern unique to carousel interfaces; (ii) click-through-rates conditioned on examination indicate that the well-known examination hypothesis does not hold in carousel interfaces; and (iii) contrary to the assumptions of previous work, users generally ignore carousel headings and focus directly on the content items. Our findings show that many user behavior assumptions, especially concerning examination patterns, do not transfer from web search interfaces to carousel recommendation settings. Our work shows that the field lacks a reliable foundation on which to build models of user behavior with these interfaces. Consequently, a re-evaluation of existing metrics and click models for carousel interfaces may be warranted. † † cc-license: by 1. Introduction With the widespread adoption of carousel interfaces by major streaming platforms such as Netflix and Spotify, these interfaces have become the de-facto standard for streaming media recommendations. They present recommendations in vertically stacked rows, where each row is a single carousel: a horizontally swipeable list of items organized around a clear theme; together, such rows form the carousel interface ( 2 ; 34 ; 24 ) , as shown in Figure 2 . Despite their ubiquitous real-world application, there is very little research on user behavior with carousel interfaces ( 28 ; 24 ; 7 ; 8 ) , especially when compared to the large amount of work on other interfaces ( 13 ; 19 ; 31 ; 35 ; 26 ) . As a result of this gap, our current understanding of how users interact with these recommendations is far from complete; and this knowledge gap significantly impedes the development of user models, evaluation metrics, and ranking algorithms tailored for carousel interfaces. Another consequence of this gap is that the few existing studies that do address carousel interfaces have to rely on behavioral assumptions that lack empirical validation ( 29 ; 27 ; 11 ; 10 ; 12 ) . Some studies have introduced the two-dimensional NDCG (N2DCG or NDCG 2D) for carousel recommendation that adopt the golden triangle (a.k.a. F-pattern ) assumption from single-list interfaces in the web search setting ( 11 ; 12 ) . These works assume that users explore a full-page carousel interface beginning at the top-left corner, with attention decreasing both horizontally (left to right) and vertically (top to bottom). Recently, 27 proposed the first click model specifically for carousel interfaces. Their approach assumes that users examine carousels sequentially from top to bottom, identify an attractive carousel solely based on its topic label without examining the items it contains, and examine individual items only when the corresponding carousel is attractive. The model further relies on the very well-known examination hypothesis , assuming that a click occurs only if an item is both examined and attractive. These assumptions are either transferred directly from single-list interfaces in web search, are adapted from them or are simplistic heuristics. Consequently, the validity of models and metrics built on these assumptions remains uncertain. This critical gap leads to the central question of this work: Are these legacy assumptions about user behavior accurate in the carousel setting, and if not, what does real user behavior with carousel interfaces actually look like? To address this research gap, we perform a detailed analysis based on a recently-released dataset from a dedicated eye-tracking study for carousel interfaces ( 7 ) . By systematically comparing the behavioral assumptions underlying current models with observations of empirical user data, we found no strong evidence to support their similarities; instead, we found significant differences between the assumptions and the data. Our main contributions are: • We introduce a L-shaped examination pattern, revealed when conditioning on clicks. We further demonstrate that the traditional F-pattern does not hold for horizontal swiping. Thus, we have two findings that highlight unique aspects of examination behavior with carousel interfaces. • We re-evaluate the examination hypothesis by estimating the conditional probability of clicking given examination. Our empirical observations heavily imply the standard examination hypothesis does not apply to carousel interfaces. • We challenge vertical navigation assumptions by showing that users largely ignore carousel topic labels and go directly to their content, contradicting the navigation logic assumed by existing carousel click models. Our findings reveal an urgent need to rethink the behavioral assumptions for carousel recommendations and provide valuable insights for building more realistic, behavior-based models and metrics that can truly capture user experiences. 2. Related Work In modern information retrieval (IR) research, the analysis of user behaviors (e.g., click logs ( 3 ; 6 ) , dwell times ( 23 ; 22 ; 35 ) , and gaze data ( 19 ; 13 ; 35 ) ) has greatly informed our understanding of how users examine and interact with search results, which in turn has shaped many of the core assumptions underlying widely used click models ( 6 ; 9 ; 16 ; 3 ; 36 ) and evaluation metrics ( 37 ; 17 ; 18 ) . 2.1. User behavior in non-carousel interfaces 2.1.1. Single-list interface The single-list interface is the most classical and extensively studied interface in IR ( 16 ; 3 ; 5 ; 13 ) . A large body of user behavior analysis based on eye-tracking and click logs has revealed a consistent examination pattern in single-list settings, known as the golden triangle , sometimes also referred to as the F-pattern ( 14 ) . It is characterized by a triangle or F-shape that indicates users concentrate heavily on the top-left area of the page, i.e., see Figure 1 . Figure 1. Visual attention heatmap of a Google results page released by Enquiro, EyeTools and Did-It. Moreover, eye tracking studies have motivated the examination hypothesis , which assumes that a document must be examined before being clicked and that click probabilities are a product of examination and attractiveness probabilities ( 30 ) . Based on this hypothesis, concrete click models introduce different additional assumptions from observed user behaviors. For example, inspired by the strong click decay across position from top to bottom ( 19 ; 13