Short-Form Video Recommendations with Multimodal Embeddings: Addressing Cold-Start and Bias Challenges

Andrii Dzhoha ∗, Katya Mirylenka ∗, Egor Malykh ∗, Marco-Andrea Buchmann, Francesca Catino · 2025

Read Short-Form Video Recommendations with Multimodal Embeddings: Addressing Cold-Start and Bias Challenges by Andrii Dzhoha ∗, Katya Mirylenka ∗, Egor Malykh ∗, Marco-Andrea Buchmann, Francesca Catino (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-19346.txt.

Introduction

Andrii Dzhoha and co-authors report lessons from launching short-video feeds inside a shopping platform, in a 2025 paper. Feeds that serve one video at a time create strong position bias, and watch-time goals skew picks toward short clips. New feeds also start with almost no user history, the classic cold-start problem.

Rather than train a standard ranking model, the team leaned on search through a tuned vision-language model that reads each video's sights and sounds. Even with plenty of viewing data on hand, that retrieval route beat supervised ranking in their tests. It also dodged some bias loops that feeds tend to feed on. The paper gives product teams a tested playbook for cold-start video feeds.

Catalog record

CREATORAndrii Dzhoha ∗, Katya Mirylenka ∗, Egor Malykh ∗, Marco-Andrea Buchmann, Francesca Catino
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE22 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2507-19346

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Short-Form Video Recommendations with Multimodal Embeddings: Addressing Cold-Start and Bias Challenges DOI: XXXXXXX.XXXXXXX Thanks: ∗ * These authors contributed equally to this work. CCS: Information systems Recommender systems Andrii Dzhoha ∗, Katya Mirylenka ∗, Egor Malykh ∗, Marco-Andrea Buchmann, Francesca Catino Abstract . In recent years, social media users have spent significant amounts of time on short-form video platforms. As a result, established platforms in other domains, such as e-commerce, have begun introducing short-form video content to engage users and increase their time spent on the platform. The success of these experiences is due not only to the content itself but also to a unique UI innovation: instead of offering users a list of choices to click, platforms actively recommend content for users to watch one at a time. This creates new challenges for recommender systems, especially when launching a new video experience. Beyond the limited interaction data, immersive feed experiences introduce stronger position bias due to the UI and duration bias when optimizing for watch-time, as models tend to favor shorter videos. These issues, together with the feedback loop inherent in recommender systems, make it difficult to build effective solutions. In this paper, we highlight the challenges faced when introducing a new short-form video experience and present our experience showing that, even with sufficient video interaction data, it can be more beneficial to leverage a video retrieval system using a fine-tuned multimodal vision-language model to overcome these challenges. This approach demonstrated greater effectiveness compared to conventional supervised learning methods in online experiments conducted on our e-commerce platform. 1. Introduction Short-form video platforms have rapidly reshaped digital engagement, with users now spending substantial time consuming immersive, vertically-scrolled video feeds. This paradigm shift has prompted established domains, including e-commerce, to experiment with similar experiences in order to capture user attention and drive engagement. However, the success of these platforms is not solely attributed to the content itself, but also to a distinctive UI innovation: rather than presenting users with a list of options, the system actively curates and presents content one item at a time, creating a highly engaging, lean-back experience. This interaction model introduces new challenges for recommender systems. Unlike traditional settings where users’ preferences are inferred from explicit choices among many options, immersive feeds rely on implicit signals such as watch-time and scroll behavior. The sequential, single-item presentation amplifies position bias ( Liu et al. 2023 ) , as users are more likely to engage with content shown earlier in the feed. Moreover, optimizing for watch-time can introduce a strong duration bias, with models tending to favor shorter videos that are more easily completed ( Zhan et al. 2022 ; Zheng et al. 2022 ) . These biases are further reinforced by feedback loops ( Chen et al. 2023 ) , making it difficult to ensure fair and relevant recommendations ( Ge et al. 2020 ) , especially in the early stages of a new product where interaction data is limited. Traditional recommender models trained from scratch, such as collaborative filtering and other supervised approaches ( Yang et al. 2020 ; Celikik et al. 2024 ; Covington et al. 2016 ) , are effective in mature platforms with plenty of representative data. However, they often struggle when available data is limited or exhibits strong biases, as is common in new product experiences. Counterfactual learning and bias correction methods have been proposed ( Joachims et al. 2017 ) , but they require careful design and large-scale data, which are often unavailable in new experiences. Mitigating such biases is challenging ( Dzhoha et al. 2024 ) , as existing methods often fail to remain robust when data is sparse ( Wang et al. 2016 ) , can show high variance in their results ( Saito et al. 2020 ) , or are affected by interleaving biases ( Vardasbi et al. 2020 ) . As a result, there is a growing need for approaches that leverage the generalization capabilities of foundation models and can be tailored for specific applications, such as launching a new short-form video experience on an e-commerce platform. In this work, we highlight the challenges encountered when launching a new short-form video experience in e-commerce. Even with access to video interaction data, conventional methods can be influenced by duration and position biases, which may limit their effectiveness. To address these issues, we present a scalable retrieval system based on a multimodal vision-language model (CLIP ( Radford et al. 2021 ) ) that maps both user history and video content into a shared semantic space. This approach leverages the generalization capabilities of foundation models, enabling robust recommendations even in cold-start scenarios and outperforming conventional supervised learning methods in our context. To personalize recommendations, we apply few-shot learning to the CLIP model using interaction data from the main e-commerce catalog, specifically from users’ Browse and Search activities, capturing nuanced user preferences. For evaluation, we utilize the expert visual language model Qwen as an LVLM-as-a-judge. Our online experiments demonstrated that this approach increased watch-time completion rates by over 39%, while maintaining a balanced distribution of video duration, popularity, and watch-time. Our main contributions are: • We discuss the unique position and duration biases in immersive short-form video feeds, explaining their impact on recommender system performance. • We share practical lessons from launching a new immersive video product, highlighting real-world challenges and the limitations of standard approaches. • We present a scalable multimodal retrieval method using vision-language models and LLM-guided evaluation and few-shot learning, which delivers improved personalization and relevance. 2. Background A recency-based solution is often the preferred initial method for video feeds, as it requires minimal engineering effort and enables rapid prototyping and launch. By surfacing the latest content, it drives engagement and content discovery, particularly in dynamic environments. This approach also minimizes the introduction of biases and the impact of feedback loops, making it a strong, interpretable baseline for evaluating more advanced personalized or multimodal methods. Once interaction data becomes available, personalization is typically introduced using a conventional two-tower architecture ( Yang et al. 2020 ; Covington et al. 2016 ; Dzhoha et al. 2025 ) , which has become standard for scalable candidate generation. These models learn separate user and item embeddings for efficient retrieval. Given the limitations of early video interaction data, a pragmatic approach is to use non-trainable user embeddings from existing platform models, while training the video tower from scratch on video interactions. Relevance is often defined by videos achieving watch times above a 50% threshold, following industry standards; however, this introduces a bias toward shorter videos ( Zhan et al. 2022 ; Zheng et al. 2022 ) . Combined with position bias and feedback loops, the resulting data can be challenging, especially in immersive feed experiences where position bias is amplified ( Liu et al. 2023 ) . Training typically uses dot-product similarity with sigmoid activation, optimized via binary cross-entropy loss, and evaluated using AUC and NDCG metrics. Other approaches include reinforcement learning for optimizing retention and watch-time ( Cai et al. 2023 ) , real-time reranking ( Gong et al. 2022 ) , and systems designed for explicit video feedback ( Lin et al. 2022 ) . While these methods increase modeling complexity or

Read the complete text