HotComment: A Benchmark for Evaluating Popularity of Online Comments CCS: Human-centered computing Social media CCS: Computing methodologies Natural language generation CCS: Computing methodologies Machine learning CCS: Computing methodologies Modeling and simulation Yafeng Wu, Yunyao Zhang, Liliang Ye, Guiyi Zeng, Junqing Yu, Chen Xu, Zikai Song Abstract . Online comments play a crucial role in shaping public sentiment and opinion dynamics on social media. However, evaluating their popularity remains challenging, not only because it depends on linguistic quality, originality, and emotional resonance, but also because stylistic preferences vary widely across platforms and user groups, causing the same comment to resonate differently in different communities. In this work, we present HotComment , a multimodal benchmark integrating video and text modalities that comprehensively quantifies popularity from three enhanced aspects: (1) Content Quality, which evaluates semantic similarity with ground-truth human comments and extends quality assessment through four interpretable dimensions; (2) Popularity Prediction, based on trends from models trained on real-world interaction data; and (3) User Behavior Simulation, which models the distribution of platform users and approximates engagement scores through an agent-based framework. Furthermore, we propose StyleCmt , inspired by social ripple effects , where multiple stylistic dimensions align to amplify socially resonant expressions and suppress incongruent ones. Figure 1 . Example from the HotComment benchmark. Illustration of three types of comments for a given video: a real human-generated popular comment from the online platform, a StyleCmt-enhanced LLM-generated comment , and a standard LLM-generated comment without stylistic guidance. 1. Introduction Online comments play a central role in shaping discourse on social media platforms ( 17 ; 89 ; 79 ) . With the rapid advancement of Artificial Intelligence and multimodal technologies ( 27 ; 63 ; 44 ; 46 ; 13 ) , the landscape of popular content generation on social media has shifted from being purely human-driven to increasingly dominated by AI-generated content ( 90 ; 88 ) . However, evaluating their popularity remains challenging ( 7 ) . Popular comments are not determined by content relevance alone, but also by stylistic expression ( 35 ) , context-aware wording ( 51 ) , and the social pathways through which audiences encounter and react to content ( 2 ; 1 ) . Existing evaluation frameworks for social media comments mainly follow two directions. One line relies on lexical or semantic similarity metrics, such as BLEU and BERTScore ( 57 ; 85 ) , to measure closeness to human references. Another extends evaluation from stylistic perspectives, incorporating factors such as humor, rhetorical devices, and creativity to better approximate human preference ( 9 ; 11 ; 93 ) . Recent benchmark efforts further emphasize multidimensional stylistic quality in comment-related generation settings ( 35 ) . While these methods capture important aspects of comment quality, they still provide only a partial account of popularity. Similarity-based metrics mainly reflect textual overlap or semantic closeness, whereas stylistic indicators are often treated as intrinsic properties of the comment itself. In practice, however, socially valued comments also depend on reasoning quality, constructiveness, and audience-sensitive engagement cues ( 23 ; 32 ; 61 ; 19 ) . Moreover, audience preferences vary across user groups, and exposure itself is shaped by demographic and cultural differences ( 21 ; 33 ; 62 ) . As a result, treating popularity as a universal textual property makes it difficult for existing frameworks to capture real engagement dynamics and the heterogeneous mechanisms underlying comment popularity ( 71 ; 20 ) . To address these challenges, we propose HotComment , a benchmark for evaluating the popularity of online comments. HotComment introduces a novel three-dimensional evaluation framework: (1) Content Quality , which evaluates semantic similarity with ground-truth human comments and additionally incorporates four stylistic dimensions ([Linguistic Expression], [Creative Imagination], [Emotional Resonance], and [Social and Cultural Influence]) to complement overlap-based metrics and capture how linguistic artistry, creativity, affective depth, and cultural propagation jointly relate to popularity; (2) Popularity Prediction , based on scores from engagement prediction models trained on large-scale real-world interaction data; and (3) User Behavior Simulation , which models heterogeneous user preferences through agent-based simulations of engagement scores . This multi-perspective design offers a more realistic and interpretable assessment of whether generated comments possess true popularity potential, establishing a robust foundation for future research on popularity-aware generation. Inspired by the Wave Interference Model ( 25 ) and the Uses and Gratifications Theory ( 30 ) , we propose StyleCmt , a novel framework that models the interaction among stylistic elements in linguistic space based on the principles of constructive and destructive interference. StyleCmt captures how different expressive patterns combine to amplify socially resonant forms while attenuating less compatible ones, generating comments that align with collective audience preferences. Experimental results demonstrate that this framework enables models to produce comments that more closely reflect human preference tendencies. Benchmark Multi Scale Content Quality Cross Aud. -modal Vis. Txt. LE CI ER SCI -plat. Var. TalkFunny ( 11 ) ✗ – 4k ✓ ✗ ✓ ✗ ✗ ✗ Chumor 2.0 ( 24 ) ✗ – 3k ✓ ✗ ✓ ✗ ✗ ✗ Puns ( 73 ) ✗ – 2k ✓ ✓ ✗ ✗ ✗ ✗ Oogiri-GO ( 93 ) ✓ 100k 30k ✓ ✓ ✗ ✗ ✗ ✗ NYT-Captions ( 26 ) ✗ 3k – ✗ ✗ ✗ ✗ ✗ ✗ ViCo ( 69 ) ✓ 20k – ✓ ✗ ✓ ✗ ✗ ✗ HOTVCOM ( 10 ) ✓ 93k – ✓ ✓ ✓ ✓ ✗ ✗ GODBench ( 35 ) ✓ 67k – ✓ ✓ ✓ ✓ ✗ ✗ HotComment ✓ 34k 47k ✓ ✓ ✓ ✓ ✓ ✓ Table 1. Benchmark comparison. Vis. and Txt. denote Visual and Text . LE , CI , ER , and SCI denote the four stylistic dimensions of Content Quality . Cross-plat. indicates datasets supporting cross-platform evaluation, and Aud. Var. denotes those considering audience variation. Our contributions are summarized as follows: • We introduce a new large-scale and comprehensive online comment dataset with a multidimensional evaluation framework for online comments tasks. • We propose StyleCmt, which simulates the interactions among different stylistic patterns, thus enabling the model to produce comments that resonate with dominant user preferences. • Extensive experiments on the HotComment benchmark demonstrate that StyleCmt effectively captures stylistic and social dynamics, significantly improving the realism and engagement alignment of generated comments compared with baselines. 2. Related Work 2.1. Evaluation of comment quality Early research on comment evaluation mainly focused on intrinsic textual quality, emphasizing coherence, fluency, and semantic consistency ( 84 ; 28 ; 12 ) with human-written references. Lexical and semantic metrics such as BLEU ( 57 ) , ROUGE ( 48 ) , and BERTScore ( 85 ) have been widely adopted to measure textual overlap and meaning alignment. Subsequent studies extended this line of work by incorporating stylistic and rhetorical dimensions into comment assessment. Existing research has examined humor ( 9 ; 11 ) , irony ( 50 ) , creativity ( 93 ) , puns ( 68 ; 73 ) , emotional expressiveness, and broader multidimensional stylistic quality ( 35 ) . These efforts deepen the understanding of how rhetorical and expressive features contribute to perceived comment quality. Beyond intrinsic quality and stylistic sophistication, another work has studied thoughtful and constructive comments as a distinct form of socially valued response. These studies show that constructiveness, reasoning quality, and conversational usefulness are not always equivalent to raw popularity feedback such as likes or replies, and should often be modeled separately ( 23 ; 32 ; 19 ; 31 ; 61 ) . Studies on constructive comments show that constructiveness, reasoning quality, and conversational usefulness often diverge from raw popularity signals like likes or replies, and should be modeled separately ( 23 ; 32 ; 19 ; 31 ; 61 ) . A similar mismatch occurs in image retrieval, where superficial engagement cues do not reliably reflect true relevance or user utility ( 18 ; 45 ; 47 ; 43 ) . These converging findings suggest that valuable feedback must often be disentangled from coarse popularity metrics. Overall, existing comment evaluation methods mainly focus on intrinsic textual and stylistic quality, providing limited support for modeling platform-dependent engagement patterns and heterogeneous audience preferences. To address this gap, HotComment extends evaluation beyond the comment itself by introducing two complementary dimensions: popularity prediction and user behavior simulation . 2.2. Comment generation Early studies on automatic comment generation ( 92 ; 59 ; 53 ) mainly relied on deep learning frameworks based on attention mechanisms ( 38 ; 78 ) , structured modeling ( 91 ; 86 ; 66 ) , or unsupervised matching between articles and comments. Later approaches improved diversity and structural representation through graph-based and retrieval-augmented designs ( 40 ; 76 ; 77 ) , but they remained limited in semantic depth and rhetorical control. With the rise of large language models ( 5 ; 4 ; 6 ; 37 ) , comment generation has advanced substantially in multi-modal settings ( 69 ; 10 ; 41 ; 42 ) , include both both text and visual cues ( 67 ; 64 ; 39 ; 65 ) . Recent studies have explored personalized comment generation ( 81 ; 49 ) , popularity-aware social response generation ( 80 ) , and controllable style steering during inference ( 83 ) , showing that user identity, stylistic preference, and anticipated audience response are important conditioning factors. Existing methods, however, typically improve only a specific aspect of generation, such as creativity ( 93 ) , humor ( 11 ) , or contextual relevance ( 69 ) . In the multimodal setting, increasingly realistic tasks and datasets, such as LiveBot ( 52 ) , PLVCG ( 82 ) , knowledge-enhanced live video comment generation ( 8 ) , and MMLSCU ( 54 ) , have further enriched the problem setting. Nevertheless, most existing methods still treat controllable factors such as style, humor, or preference as isolated attributes, rather than explicitly modeling how multiple stylistic tendencies interact within a comment community and jointly shape audience resonance. To address this limitation, we propose StyleCmt , a cross-platform and multimodal framework that models comment-section preferences and generates comments aligned with dominant audience styles and expectations. 3. Challenge and Motivation Challenge. Evaluating the popularity of online comments is more challenging than assessing intrinsic text quality alone. In social media, popularity is not a fixed property of the comment itself, but a conditional outcome jointly shaped by the comment, the source content, the platform context, and the exposed audience: (1) \pi(c\mid x,p,a) where c denotes the comment, x the source content, p the platform context, and a the audience state. This formulation makes explicit that popularity depends on both contextual conditions and audience structure. However, existing benchmarks and evaluation protocols mainly focus on semantic similarity, fluency, or isolated stylistic aspects, with limited ability to capture how these factors interact with real engagement mechanisms. A single popularity predictor reduces dissemination to a coarse platform-level estimate , while generic judge-style evaluation ignores exposure conditions, audience heterogeneity, and variation in user responses. Recent in-the-wild evidence from public human–LLM interactions on social media further shows that engagement in multi-party environments is highly asymmetric and socially embedded, making popularity inseparable from exposure context and audience structure ( 55 ) . As a result, current evaluation settings remain insufficient to determine whether a generated comment truly has real-world popularity potential. Motivation and Design Rationale. These limitations motivate a benchmark design that approximates popularity from multiple complementary perspectives: • Content Quality evaluates whether a generated comment is semantically appropriate and stylistically expressive, providing the textual foundation of popularity. • Popularity Prediction captures platform-level interaction trends learned from real-world engagement data, reflecting aggregate platform tendencies. • User Behavior Simulation models heterogeneous audience responses at the user level. Rather than acting as a generic LLM judge, it serves as a data-grounded audience modeling component . User profiles and agent weights are allocated according to real-world demographic statistics, regional internet population characteristics, and platform-specific user category distributions, ensuring that simulated reactions are conditioned on realistic audience composition. Together, these three dimensions provide a structured approximation of popularity under heterogeneous exposure conditions, avoiding the reduction of complex popularity mechanisms to a single score and yielding a more faithful evaluation framework for socially grounded comment generation. Figure 2 . HotComment evaluates models from three key aspects:(1) Content Quality , assessed through multi-dimensional semantic comparison with top- k popular comments in the dataset; (2) Popularity Prediction , based on trends from popularity prediction models trained on real-world interaction data; (3) User Behavior Simulation , conducted via agent-based modeling of online user behavior such as thumbs-up. 4. HotComment Benchmark We first compare HotComment with the previous benchmark in Tab. 1 . Then, we introduce the three core dimensions of HotComment : Content Quality , which assesses linguistic and stylistic express; Popularity Prediction , which estimates engagement likelihood based on real-world interaction data; and User Behavior Simulation , which models audience preferences and interaction patterns to reflect realistic social dynamics. 4.1. Dataset Construction Data Sources We construct the HotComment dataset by collecting large-scale article–comment and video–comment pairs from multiple mainstream online platforms, including NetEase News, Tencent News, and Bilibili. These sources cover both textual and audiovisual content, enabling the study of multimodal communicative behavior and stylistic variation across platforms. Scale and Composition The dataset comprises over 43,000 online articles and 34,000 videos , encompassing approximately 1.4 million content-comment pairs in total. Each article or video is paired with corresponding user comments that reflect diverse linguistic styles and engagement behaviors. For each item, we collect both popular comments with high user engagement and non-popular comments with relatively lower engagement; the average number of comments per item varies by content type and platform characteristics. Popularity Labeling High-quality (popular) comments are defined as those ranked within the top-15 by likes , with like counts exceeding either 10% of the top comment or an absolute threshold of 2,000 likes . For low-quality (non-popular) comments, we select those posted on the same day as the content with like counts not exceeding 10 , collecting at most 10 such comments per item. This hybrid criterion ensures balanced representation while mitigating temporal bias. 4.2. Evaluation Methods 4.2.1. Content Quality To comprehensively assess the intrinsic quality of generated comments, we evaluate not only their semantic similarity to human-written references but also their stylistic expressiveness ( 35 ) across four complementary dimensions: • Linguistic Expression evaluates the rhetorical and writing artistry of comments, focusing on linguistic creativity such as humor, irony, metaphor, rhythm, and aesthetic fluency that enhance expressiveness and readability. • Creative Imagination measures the degree of originality and associative thinking within comments, capturing the ability to connect distant concepts or construct unexpected, imaginative scenarios that extend semantic boundaries. • Emotional Resonance examines the emotional depth and attitudinal stance of a comment, emphasizing its capacity to evoke empathy, convey genuine sentiment, or express strong affective tones that engage readers. • Social and Cultural Influence assesses the comment’s potential for social propagation, including the use of memes, cultural references, and intertextual expressions that facilitate sharing, imitation, and collective resonance across communities. 4.2.2. Popularity Prediction This dimension models platform-level engagement preferences to estimate how likely a comment would become popular within a given social media environment. Different platforms exhibit distinct user cultures and interaction mechanisms, leading to diverse definitions of “popularity”. To reflect these differences, we train an individual prediction model for each platform using a fine-tuned BERT encoder with a task-specific MLP head, with real interaction data converted into binary popularity labels as supervision signals. Each model jointly encodes the contextual information of the article or video and the associated comment, and outputs a predicted popularity score that reflects engagement likelihood. Recent benchmark construction for social-media popularity prediction has also begun to emphasize temporal alignment and temporal dynamics, suggesting that engagement modeling should consider not only content–comment matching but also time-sensitive propagation patterns ( 72 ) . Training is performed under a binary classification objective, where popular and non-popular comments serve as positive and negative samples, respectively. We employ a combination of cross-entropy loss to ensure accurate classification and a supervised contrastive loss to enhance representation discrimination between high- and low-engagement comments. This setup enables the model to learn platform-specific engagement patterns while providing a reliable data-driven metric for evaluating the real-world popularity potential of generated comments. This design is broadly consistent with prior studies on popularity prediction and diffusion modeling, which treat engagement as a function of structural spread patterns, interaction dynamics, or jointly encoded content signals ( 14 ; 22 ; 3 ) . 4.2.3. User Behavior Simulation User Behavior Simulation models the composition of the exposed audience to approximate how comments would be perceived by different user groups within a realistic social environment. We formulate the exposure process as a two-level hierarchical simulation . • At the top level, we classify potential viewers into two primary categories: interested users and casual viewers . For each platform–domain pair, we first assign an Exposure Specificity Index (ESI) that represents the baseline exclusivity of audience exposure. A domain-specialized agent analyzes the content to determine audience exposure patterns based on five key determinants of selective exposure: channel verticality, distribution channel characteristics, event salience, emotional arousal level, and celebrity or authority involvement . The agent outputs an adjusted proportion, denoted as p_{I}^{*} , representing the estimated share of interested users in the total audience. This modeling approach aims to reproduce the realistic composition of audiences who actually see the content, aligning with established theories of selective exposure, incidental news contact, and high-arousal dissemination. • At the lower level, we derive the distribution of user subgroups from publicly accessible regional internet demographics and platform-specific user category statistics , rather than preset rules. These data act as data-driven priors to construct heterogeneous audience segments with distinct interaction tendencies. This hierarchical design enables a user-centered interpretation of comment popularity beyond aggregate platform metrics. Through this hierarchical design, the simulation offers a population-level perspective on how comments are likely to be received, thereby complementing platform-level popularity prediction with user-centric interpretability . Our formulation is also related to recent LLM-agent simulation frameworks, which emphasize role-conditioned behavior, social interaction, and aggregate engagement dynamics in synthetic but data-grounded online environments ( 58 ; 70 ; 36 ; 60 ) . This perspective is further supported by recent multi-agent social-media simulation work, which models public-opinion evolution through cognitively grounded agents and dynamic interaction environments ( 87 ) . Figure 3 . Overview of the StyleCmt Framework. The comment generation process is modeled as a form of wave interference in a stylistic field. The pipeline illustrates four consecutive stages: (1) Resonance Field Construction . Retrieving similar hot comments and decomposing their stylistic components across four dimensions (LE, CI, ER, SCI); (2) Interference-Driven Planning . Aggregating and identifying dominant stylistic patterns to form an interference blueprint; (3) Coherent Superposition . Generating multiple linguistic realizations under the same stylistic configuration; and (4) Collapse and Emission – selecting and refining the most resonant comment as the final output. 5. StyleCmt Framework 5.1. Motivation and Computational Grounding The StyleCmt framework is inspired by the way ideas and expressions spread through social interaction and gradually form shared preferences within a community. In our framework, historical comments under the same content context collectively define a stylistic resonance field , which captures the dominant expressive tendencies of the comment section. To model this process, we represent each stylistic dimension as a vector with two components: intensity , which reflects the strength of community preference for that style, and orientation , which encodes its semantic and affective direction. When the stylistic representation of a generated comment is consistent with dominant historical patterns, the comment exhibits stronger stylistic coherence and better matches community preference. When the alignment is weak, the generated expression is less coherent with the surrounding discourse. This formulation provides a principled way to model stylistic preference in vector space and supports the generation of comments that are more consistent with collective audience expectations. 5.2. Framework Overview and Modeling Process Resonance Field Construction. This step performs contextual style distribution modeling . To capture how a community tends to express itself, we construct the stylistic resonance field based on historical comments. Each comment is decomposed into four stylistic dimensions: Linguistic Expression, Creative Imagination, Emotional Resonance, and Social/Cultural Influence. For each dimension i , we estimate two properties from the data: (1) an intensity scalar \bar{A}_{i}(\mathbf{x}) , representing how strongly the community prefers this stylistic pattern at context \mathbf{x} , and (2) an orientation unit vector \bar{\mathbf{d}}_{i}(\mathbf{x}) , describing its typical stylistic inclination. We summarize the overall contextual stylistic tendency as a weighted vector aggregation: (2) \mathbf{\Psi}_{0}(\mathbf{x})=\sum_{i=1}^{4}\bar{A}_{i}(\mathbf{x})\,\bar{\mathbf{d}}_{i}(\mathbf{x}), which acts as a compact mathematical representation of the community’s baseline preference. Interference-Driven Planning. This phase executes alignment-driven style planning . Given the base preference \mathbf{\Psi}_{0} , we determine how a newly generated comment should adjust its stylistic mix. A candidate comment introduces a set of controllable stylistic contributions: (3) \mathbf{v}_{i}=A_{i}\,\mathbf{d}_{i}, where A_{i} denotes the intended intensity of stylistic pattern i , and \mathbf{d}_{i} describes its stylistic direction. To evaluate how well this candidate aligns with community tendencies, we compute the interaction score via the dot product of their orientation vectors (equivalent to cosine similarity): (4) I_{ij}=\mathbf{d}_{i}\cdot\bar{\mathbf{d}}_{j}. A positive value indicates that the intended direction \mathbf{d}_{i} is compatible with the community inclination \bar{\mathbf{d}}_{j} , conceptually mimicking constructive interference. The planning objective selects \{A_{i},\mathbf{d}_{i}\} that maximize the overall alignment: (5) \mathcal{E}_{\text{align}}=\sum_{i