A Computational Model of Message Sensation Value in Short Video Multimodal Features that Predicts Sensory and Behavioral Engagement

2026

Read A Computational Model of Message Sensation Value in Short Video Multimodal Features that Predicts Sensory and Behavioral Engagement (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-19995.txt.

Introduction

Xue and four colleagues build a computational model of message sensation value, a measure of how sensational short videos feel, from multimodal cues. They test the model on human ratings of 1,200 short videos, then validate it on two unseen sets from three platforms totaling 14,492 videos. Higher sensation value boosts sensory engagement, but behavior peaks at moderate levels. Stronger sensation mainly stirs the senses, giving researchers a robust tool for short-video study.

Catalog record

YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE50364
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.19995

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

A Computational Model of Message Sensation Value in Short Video Multimodal Features that Predicts Sensory and Behavioral Engagement Abstract The contemporary media landscape is characterized by sensational short videos. While prior research examines individual multimodal features’ effects, the collective impact of multimodal features on viewer engagement with short videos remains unknown. Grounded in the theoretical framework of Message Sensation Value (MSV), this study develops and tests a computational model of MSV with multimodal feature analysis and human evaluation of 1,200 short videos. This model that predicts sensory and behavioral engagement was further validated across two unseen datasets from three short video platforms (combined N = 14,492). While MSV is positively associated with sensory engagement, it shows an inverted U-shaped relationship with behavioral engagement: Higher MSV elicits stronger sensory stimulation, but moderate MSV optimizes behavioral engagement. This research advances the theoretical understanding of short video engagement and introduces a robust computational tool for short video research. A Computational Model of MSV \authorsnames Haoning Xue, Jingwen Zhang, Xiaohui Wang, Diane Dagyong Kim, Yunya Song \authorsaffiliations Department of Communication, University of Utah, Department of Communication, University of California, Davis, Department of Media and Communication, City University of Hong Kong, Department of Communication, University of California, Davis, Division of Emerging Interdisciplinary Areas, Hong Kong University of Science and Technology keywords Video-As-Data, Message Sensation Value, Short Video Analytics, Social Media Engagement 1 Introduction Short video platforms have become important information sources, but how individuals process multimodal features remains unclear. Recent studies have been examining individual multimodal features in short videos, such as brightness ( 30 ) and audio tempo ( 27 ) . However, since individuals process multimodal information collectively, not in isolation, existing findings on single multimodal features’ effects fall short of capturing the complex, integrated nature of multimodal information processing. This underscores a key research gap: A unified, theory-driven approach to understanding the holistic effect of multimodal features ( 24 ) . Message Sensation Value (MSV) provides a useful theoretical framework for addressing this challenge. MSV quantifies the extent to which objectively measured message features and audiovisual content attract attention and elicit affective responses ( 40 ) . MSV influences individual attention allocation by activating the neural system and triggering the sensory processing of information ( 34 ) . However, competing theoretical arguments and empirical evidence exist on how exactly MSV influences attention. Theories grounded in communication and cognitive science, such as the Activation Model of Information Exposure (AMIE; 9 ) , the Limited Capacity Model of Mediated Message Processing (LC4MP; 26 ) , and the Elaboration Likelihood Model (ELM; 43 ) , have argued for opposite mechanisms. AMIE and LC4MP propose that MSV functions as an attention-getter, whereas ELM argues for an attention-distracting role. Both arguments have received empirical support, and a more precise examination of the theoretical mechanism of MSV’s effects requires a reliable and scalable predictive model. Grounded on the literature on MSV, Morgan and colleagues (2003) developed a measure of MSV by manually annotating formal (e.g., shot changes) and content (e.g., surprise ending) features. While foundational, this objective MSV measure that was contextualized in televised Public Service Announcements (PSAs) needs to be adapted to the contemporary media environment. The short video format is very different from PSA’s long video format, so many features selected based on PSAs may not be relevant (e.g., formats unexpected in PSAs) or be quantifiable by existing computational models (e.g., surprise end). Distinctive from PSAs, short video formats are used across a broader range of content beyond science and health topics, created and distributed by both ordinary users and institutions, consumed within algorithm-curated video feeds rather than broadcast television, and engaged with interactive social media features (e.g., likes, comments, shares) beyond viewing alone. The combination of content diversity ( 51 ) , algorithm curation ( 67 ) , and real-time engagement features (e.g., Danmu in 60 ) makes short video processing and engagement dynamics much more complex than the traditional PSA engagement processes from the mass media era. Thus, it necessitates the reinvestigation of MSV and its influences on sensational experiences and behavioral engagement. Further, on the methodological front, the past research on MSV used traditional content analyses involving labor-intensive and manual annotation processes. The approach of using binary annotation of features also oversimplifies continuous audiovisual dynamics. These limitations highlight the need for a robust and scalable model that automatically predicts the holistic sensory impact of multimodal features. The present study addresses this gap by developing a computational model that (a) quantifies the MSV of multimodal features in short videos and (b) predicts viewer engagement. Short videos tend to be designed and prioritized by platforms to attract more rapid, sensation-driven attention. This focus aligns with the literature on MSV and sensory message processing since attention is one of the primary responses elicited by MSV ( 10 ) . Unlike traditional PSA messages communicated one-way on TV, the engagement with short videos is more dynamic and collectively shaped by individual users and recommendation algorithms. We conceptualize video engagement along two dimensions, following the two-stage model of attention ( 12 ) . First, sensory engagement captures viewers’ immediate, affective attention to sensational videos, which is measured with Perceived Message Sensation Value (PMSV). It reflects an individual’s subjective perception and evaluation of a message in terms of the extent of emotional arousal, novelty, and dramatic impact ( 41 ) . PMSV here serves as a subjective proxy for how sensational a video feels to viewers. Second, behavioral engagement represents viewers’ actions toward a video (e.g., likes, comments, shares) in real media environments. Engagement metrics reflect a higher level of engagement when viewers are activated to like, comment on, and share a video ( 12 ) . These two dimensions capture a fuller process of how short videos attract attention and translate sensory stimulation into behavior activation. This study used (a) computational video analysis to extract 20 multimodal features from N = 1200 short videos on Instagram Reels across eight critical societal issues and (b) machine learning models to pinpoint the combination of relevant multimodal features that best predict sensory engagement (i.e., PMSV; N_{\text{participant}} = 1007). We focus on sensory engagement rather than behavioral engagement for model development because PMSV reflects human perceptions of a video’s capacity to elicit sensory experience, which is the most theoretically aligned and empirically measurable approximation of MSV. Specifically, PMSV measures how sensational a video is perceived by viewers, whereas MSV refers to the message properties that give rise to that perception. This choice is also supported by solid empirical evidence on a strong relationship between MSV and PMSV ( 35 ) . Therefore, using PMSV as an established measure of sensory experiences to back-develop the computational MSV model is both theoretically and empirically justified. The contribution of this research is threefold. First, the findings contribute to the ongoing debates on the attentional mechanism of MSV by presenting correlational evidence of an inverted U-shaped curve between MSV and public engagement. This finding suggests that MSV may attract attention up to a point, highlighting the potential importance of the “middle ground” of MSV and advocating for a more granular examination of MSV in future research. Second, we provide a scalable and theory-informed computational model of

Read the complete text