Counting How the Seconds Count: Understanding Algorithm-User Interplay in TikTok via ML-driven Analysis of Video Content
Maleeha Masood, Shreya Kannan, Zikun Liu, Deepak Vasisht, Indranil Gupta · 2025
Read Counting How the Seconds Count: Understanding Algorithm-User Interplay in TikTok via ML-driven Analysis of Video Content by Maleeha Masood, Shreya Kannan, Zikun Liu, Deepak Vasisht, Indranil Gupta (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-20030.txt.
Introduction
Maleeha Masood and co-authors study how TikTok's feed and its users shape each other over time. Their 2025 paper adds a fresh lens: software that reads the videos themselves, not just titles and tags. A new tool uses vision-language models to code what each clip shows.
The team pairs that tool with user studies, interviews, and donated watch histories. They track how well users think the feed fits them, and how their taps steer later picks. Order effects show up too, since the same clips land unlike when shuffled. The authors also probe whether feed quality can be forecast. Their goal is insight for the user community, not a copy of the secret algorithm.
Catalog record
| CREATOR | Maleeha Masood, Shreya Kannan, Zikun Liu, Deepak Vasisht, Indranil Gupta |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 100 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2503-20030
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Counting How the Seconds Count: Understanding Algorithm-User Interplay in TikTok via ML-driven Analysis of Video Content Maleeha Masood, Shreya Kannan, Zikun Liu, Deepak Vasisht, Indranil Gupta Abstract . Short video streaming systems such as TikTok, YouTube Shorts, Instagram Reels, etc., have reached billions of active users worldwide. At the core of such systems are (proprietary) recommendation algorithms which recommend a sequence of videos to each user, in a personalized way. We aim to understand the temporal evolution of recommendations made by such algorithms, as well as the interplay between the recommendations and user experience. While past work has studied recommendation algorithms using textual data (e.g., titles, hashtags, etc.) as well as user studies and interviews , we add a third modality of analysis—we perform automated analysis of the videos themselves . To perform such multimodal analysis, we develop a new HCI measurement approach that starts with our new tool called VCA (Video Content Analysis) that leverages recent advances in Vision Language Models (VLMs). We apply VCA on a trifecta of HCI methodologies—real user studies, interviews, and data donation. This allows us to understand temporal aspects of how well TikTok’s recommendation algorithm is perceived by users, is affected by user interactions, and aligns with user history; how users are sensitive to the order of videos recommended; and how the algorithm’s effectiveness itself may be predictable in the future. While it is not our goal to reverse-engineer TikTok’s recommendation algorithm, our new findings indicate behavioral aspects that the TikTok user community can benefit from. Conference: Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems; April 13–17, 2026; Barcelona, Spain Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems (CHI ’26), April 13–17, 2026, Barcelona, Spain DOI: 10.1145/3772318.3790311 ISBN: 979-8-4007-2278-3/2026/04 CCS: Human-centered computing Social media CCS: Computing methodologies Artificial intelligence † † cc-license: by 1. Introduction We are seeing an explosion in the popularity of short video streaming applications over the past few years. TikTok, one of the leading short video platforms, reports more than 1 billion users worldwide, with an estimated 82 million active users daily in the US in 2025 ( 1bi, 2025 ) . On average, users on TikTok spend nearly an hour per day on the app ( Duarte, 2024 ) . Other major social media platforms have also released short video streaming services like Instagram Reels, Facebook Watch and YouTube Shorts. As a result, the overall adoption of short video applications continues to grow worldwide ( pop, 2024b ; pop, 2024a ; pop, 2023 ) . On TikTok, the “For You Page” is an ongoing algorithmic experience between the short video streaming platform and a user, consisting of a sequence of videos appearing to be uniquely personal. Central to this experience is a recommendation algorithm, which generates the user-specific video recommendations. A user watches these videos in the order shown, although they can always skip to the next video by swiping. These algorithms require little explicit input about user preferences, but rather learn about interests through interactions. The recommendation algorithms play a key role in the success of such platforms, but these algorithms remain proprietary. Existing knowledge about how recommendation algorithms function and interact with users has been largely derived from qualitative accounts of users’ personal experiences with short video streaming systems and ultimately, user - constructed folk theories. Prior human-computer interaction research ( Rader and Gray, 2015 ; Eslami et al., 2016 ; DeVito et al., 2018 ; Karizat et al., 2021 ; DeVito, 2021 ; Mayworm et al., 2024 ) has found that users develop improvised folk theories (explanatory frameworks, often partial) to make sense of the recommendation algorithm. These user-generated explanations, in turn, shape everyday practices: users attempt to engage in “training” the algorithm by strategically liking, skipping, sharing, searching and commenting on videos, and the user interprets subsequent changes in their feed as personalized responses to specific actions or attributes. Such practices not only encapsulate users’ beliefs (mistaken or otherwise) about the system, but they also loop back into the platform, further entangling user behavior, algorithmic inference, and perceived control. In this work, we precisely examine how the recommendation system behaves as it interacts with a user over time, by shifting focus from the qualitative experiences of a user to the quantitative behavior of the recommendation system itself. We aim to understand the evolution of recommended content over time and the interplay between the algorithm’s output (i.e., recommended videos) and the user experience . Rather than proposing or deepening HCI theories, we offer a practical HCI measurement approach that is immediately usable by HCI researchers in this fast-moving field. Our approach enables analysis of video content at scale, which is difficult to achieve with existing methods of working with multimodal data. We also show how our HCI measurement approach enables us to unearth several findings that empirically test or challenge widely held folk theories about recommender systems, including the role of user interactions and sequence continuity. We answer the following research questions (RQs): • RQ1 How does the content of recommendations made by TikTok’s recommendation algorithm evolve over time at short and long timescales? • RQ2 What is the relationship between user interactions (likes and shares) and recommendations made by the algorithm? • RQ3 Do users engage more with recommendations that align with their historical video consumption patterns? • RQ4 Beyond the videos recommended, does the continuity of the recommended video sequence matter (in terms of user satisfaction)? • RQ5 To what extent can we predict whether a user will watch a recommended video using our video analysis pipeline? Interpreting the RQs: These RQs cover all three entities involved—provider, user, video sequence. Concretely, we study the content of video recommendations from the provider (RQ1), user interactions and preferences (RQs 2-3), and the sequence of videos recommended (RQs 4-5). Alternately, these RQs can be seen as focusing on both past and present histories (RQs 1-4), and also looking forward to the future (RQ5). A third way to see these RQs is that together they trace the life cycle of recommendations: from how they evolve over time (RQ1), how they respond to user behavior (RQ2–RQ4), and whether they affect engagement (RQ5). Other important questions (e.g., fairness or creator outcomes) are orthogonal to our scope. Existing Work, and Need for a New Approach: Recent work has attempted to analyze the emergent behaviors of TikTok’s recommendation algorithms through several techniques. First, some studies manually analyze video content ( Basch et al., 2021b ; Basch et al., 2021a ; Alley and Hanshew, 2022 ) or conduct in-person or online interviews to understand folk theories with respect to the recommendation algorithm ( Karizat et al., 2021 ; Lee et al., 2022 ; DeVito, 2022 ) . For e.g., manual analysis of 100 videos was used to study the content of climate change-related videos ( Basch et al., 2021b ) . Manual analysis requires extensive effort and cost, and is hard to scale beyond a few hundred videos. Other work ( Boeker and Urman, 2022 ; DeVito, 2022 ) relies on automatic analysis of TikTok browsing histories to conduct large-scale empirical analysis of user interactions in TikTok (e.g., finding that users more likely to ‘