Vastu : V alue- A ligned S ocial T oolkit for Online Content C u ration
Agam Goyal, Xianyang Zhan, Charlotte Lambert, Koustuv Saha, Eshwar Chandrasekharan · 2026
Read Vastu : V alue- A ligned S ocial T oolkit for Online Content C u ration by Agam Goyal, Xianyang Zhan, Charlotte Lambert, Koustuv Saha, Eshwar Chandrasekharan (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-12491.txt.
Introduction
Agam Goyal and four co authors build a toolkit for finding what online communities value. Knowing what a community values drives feed curation, ranking, moderation, and recommendation. Past ways to detect those values are split across distinct schools of method. It is unclear which school best captures the values of a given community.
Their toolkit is called Vastu, short for a value aligned social toolkit for content curation. The name signals its aim: tools that align with the values of each community. A shared toolkit lets methods be compared on the same ground. It also helps platform teams pick the right method for their users. The work sits in social computing, where values shape what people see each day.
Catalog record
| CREATOR | Agam Goyal, Xianyang Zhan, Charlotte Lambert, Koustuv Saha, Eshwar Chandrasekharan |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 63 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2601-12491
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Vastu : V alue- A ligned S ocial T oolkit for Online Content C u ration Agam Goyal, Xianyang Zhan, Charlotte Lambert, Koustuv Saha, Eshwar Chandrasekharan Abstract Detecting what content communities value is a foundational challenge for social computing systems—from feed curation and content ranking to moderation tools and personalized recommendation systems. Yet existing approaches remain fragmented across methodological paradigms, and it remains unclear which methods best capture community-specific notions of value. We introduce Vastu ( V alue- A ligned S ocial T oolkit for Online Content C u ration), a benchmark and evaluation framework for systematically comparing approaches to detecting community-valued content. Vastu includes a dataset of 75,000 comments from 15 diverse Reddit communities, annotated with community approval labels and rich linguistic features. Using Vastu , we evaluate feature-based models, transformers, prompted and fine-tuned language models under global versus community-specific training regimes. We find that community-specific models consistently outperform global approaches, with fine-tuned transformers achieving the strongest performance (0.72 AUROC). Notably, fine-tuned SLMs (0.65 AUROC) substantially outperform prompted LLMs (0.60 AUROC) despite being 100 times smaller. Counterintuitively, chain-of-thought prompting provides no benefit, and reasoning models perform the worst (0.53 AUROC), suggesting this task requires learning community norms rather than test-time reasoning. By releasing Vastu , we provide a standardized benchmark to advance research on value-aligned sociotechnical systems. 1 1 1 The code for all experiments and the toolkit will be released publicly upon acceptance in accordance with Reddit policies. 1 Introduction The volume of content on social media platforms has grown far beyond what users can navigate manually. Billions of posts, comments, and submissions are created daily, making algorithmic curation essential for determining what content surfaces to users ( 45 ; 12 ) . These systems span diverse applications, from feed curation and content ranking to recommendation engines, and proactive moderation tools that identify exemplary contributions ( 31 ) . Yet a common prerequisite underlies all such applications: the ability to reliably detect what content communities actually value . Without robust detection tools, platforms cannot surface valued content to users, moderators cannot identify exemplary contributions to encourage, and researchers cannot study the nature of community preferences at scale. Despite this shared need, existing approaches to detect community-valued content remains fragmented across methodological paradigms. Some work applied traditional machine learning ( 13 ) , others fine-tune transformer-based models ( 40 ) , and more recent works use prompted large language models (LLMs) ( 25 ; 14 ) . However, these methods for the task of value-aligned content curation have not yet been compared systematically. Several questions remain unanswered: What are the relative strengths of feature-based versus language model-based approaches? Under what conditions does each method excel or fail? What linguistic and semantic cues do these methods actually leverage, and do they align with what we know about community preferences from prior work? This fragmented understanding limits progress across the many applications that depend on value detection. Beyond methodological fragmentation, existing work also faces a conceptual tension. Recent approaches operationalize “values” through universal psychological frameworks like Schwartz’s basic human values ( 43 ) . While theoretically grounded, such approaches may fail to capture the idiosyncratic preferences that distinguish one community from another. For example, contributions valued in gaming forums differ substantially from what resonates in a scientific discussion or advice-seeking context. Other work has explored personalized recommendation ( 5 ; 33 ) , but collecting individual user preferences at scale remains challenging. We therefore need is a framework for evaluating how well different approaches capture community-specific notions of value. We address these gaps by introducing Vastu ( V alue- A ligned S ocial T oolkit for Online Content C u ration), a unified framework for evaluating approaches to detecting community-valued content. Vastu comprises of two core components. First, we contribute a benchmark dataset of 75,000 comments from 15 diverse Reddit communities spanning Gaming, Science, Q&A, Advice, and Politics—annotated with community approval labels derived from platform voting patterns and augmented with rich linguistic features. This benchmark enables standardized evaluation across modeling paradigms. Second, we provide a comprehensive evaluation methodology that compares feature-based models, fine-tuned transformers, prompted LLMs, and fine-tuned small language models under both global and community-specific training regimes. As part of Vastu , we leverage Reddit communities as natural proxies for distinct preference profiles, operationalizing community values through the platform’s voting mechanism. Different subreddits exhibit markedly different upvoting patterns due to varying participant demographics, topical focuses, and cultural norms, where each community effectively represents a distinct “user profile” with its own standards for quality content ( 14 ) . This approach has also been shown by prior work to encode emergent, context-dependent preferences rather than imposing predefined value taxonomies ( 13 ; 27 ; 37 ) . Using Vastu , we conduct comprehensive experiments addressing three core research questions: RQ1: How effectively can different modeling approaches detect community-valued content? We compare feature-based methods against language models across diverse Reddit communities. We examine how performance varies across communities and whether community-specific models outperform global approaches. RQ2: What signals do different approaches rely on to identify valued content? We apply SHAP analysis ( 32 ) to feature-based models and qualitatively code chain-of-thought explanations from language models to surface the cues each approach relies on. RQ3: Where and why do different approaches fail? We conduct systematic error analysis to identify failure modes across methods, characterizing the types of content that prove difficult to classify. Summary of Findings: Our experiments reveal that moderately-sized, community-specific models substantially outperform both global models and large language models. Fine-tuned transformers achieve the strongest performance (XLNet: 0.72 AUROC), but fine-tuned 4B-parameter language models offer competitive results (0.65 AUROC) at dramatically lower computational cost. Surprisingly, prompted LLMs—including reasoning models—consistently underperform supervised approaches, with GPT-5-mini achieving the worst results (0.53 AUROC). Our interpretability analyses reveal that feature importance varies dramatically across community types. For example, prosociality strongly predicts values in Advice communities (+0.121 SHAP) but is penalized in Gaming (-0.056) and Politics (-0.029) communities. Error analysis shows that LLMs exhibit very high conservatism ( >80\% false negative rate), systematically discarding valued content that violates its generic notions of “quality.” These findings demonstrate that detecting community-valued content requires community-specific calibration rather than universal models, and that parametric adaptation to local norms outweighs both model scale and advanced prompting strategies. By releasing Vastu , we enable researchers and practitioners to evaluate new models and approaches against a standardized benchmark, advancing the development of systems that can surface what communities value. 2 Related Work 2.1 Content Values in Online Communities Understanding what online communities value has been approached through multiple methodological lenses. Survey-based studies have developed taxonomies of community values from user perspectives ( 47 ) and moderator perspectives on what behaviors they wish to encourage ( 28 ) , revealing that qualities like prosociality, engagement, and content quality are commonly desired. Complementing these qualitative approaches, 14 used an LLM-based framework to identify values present in highly-upvoted Reddit comments at scale, uncovering values at macro, meso, and micro scales. Research on Reddit’s voting mechanisms has shown that upvotes serve as signals of community-approved discourse styles, with 37 demonstrating this for political rhetoric in political subreddits. Other computational approaches to measuring valued behavior have largely focused