A Platform for Investigating Public Health Content with Efficient Concern Classification DOI: XXXXXXX.XXXXXXX Conference: The Web Conference 2025; April 28 - May 2, 2025; Sydney, Australia ISBN: 978-1-4503-XXXX-X/18/06 CCS: Human-centered computing Interactive systems and tools CCS: Applied computin

Christopher Li, Rickard Stureborg, Bhuwan Dhingra, Jun Yang · 2025

Read A Platform for Investigating Public Health Content with Efficient Concern Classification DOI: XXXXXXX.XXXXXXX Conference: The Web Conference 2025; April 28 - May 2, 2025; Sydney, Australia ISBN: 978-1-4503-XXXX-X/18/06 CCS: Human-centered computing Interactive systems and tools CCS: Applied computin by Christopher Li, Rickard Stureborg, Bhuwan Dhingra, Jun Yang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-01308.txt.

Introduction

Christopher Li, Rickard Stureborg, Bhuwan Dhingra, and Jun Yang built a platform for studying health worries online, presented in a 2025 paper. Posts that voice concern about health measures have grown, and they slow the uptake of steps that prevent disease. Officials need tools to see what concerns people raise and how they shift.

Their platform, ConcernScope, uses a teacher-student setup: a large model teaches small, fast classifiers to tag concerns in text. Users can upload big files, pull text from web addresses, or edit text by hand. A worked example sorted 186,000 samples over time to show trends around major events. The tool is meant to help officials respond to public concerns with better timing.

Catalog record

CREATORChristopher Li, Rickard Stureborg, Bhuwan Dhingra, Jun Yang
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE33 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2506-01308

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

A Platform for Investigating Public Health Content with Efficient Concern Classification DOI: XXXXXXX.XXXXXXX Conference: The Web Conference 2025; April 28 - May 2, 2025; Sydney, Australia ISBN: 978-1-4503-XXXX-X/18/06 CCS: Human-centered computing Interactive systems and tools CCS: Applied computing Health care information systems CCS: Computing methodologies Machine learning CCS: Computing methodologies Natural language processing CCS: Information systems Information extraction CCS: Applied computing Document management and text processing Christopher Li, Rickard Stureborg, Bhuwan Dhingra, Jun Yang Abstract . A recent rise in online content expressing concerns with public health initiatives has contributed to already stalled uptake of preemptive measures globally. Future public health efforts must attempt to understand such content, what concerns it may raise among readers, and how to effectively respond to it. To this end, we present ConcernScope, a platform that uses a teacher-student framework for knowledge transfer between large language models and light-weight classifiers to quickly and effectively identify the health concerns raised in a text corpus. The platform allows uploading massive files directly, automatically scraping specific URLs, and direct text editing. ConcernScope is built on top of a taxonomy of public health concerns. Intended for public health officials, we demonstrate several applications of this platform: guided data exploration to find useful examples of common concerns found in online community datasets, identification of trends in concerns through an example time series analysis of 186,000 samples, and finding trends in topic frequency before and after significant events. 1. Introduction Understanding public concerns about health interventions is a critical challenge for public health practitioners, particularly in the domain of vaccine hesitancy. In recent years, vaccine concerns have posed growing challenges to public health as vaccine rates have stalled ( Neely et al. 2022 ; Gür 2019 ) . Identifying trends in such concerns would help inform public health campaigns to address specific topics and guide future efforts ( Stureborg 2024 ) . Prior research in this space largely focuses on detecting sentiment—positive, neutral, or negative—toward vaccines on social media ( Yousefinaghani et al. 2021 ; Kang et al. 2017 ; Yiannakoulias et al. 2019 ) . However, many of these studies do not analyze the specific themes underlying these sentiments and few track how these concerns change over time. There are also formal reporting systems and social listening tools within the public health field, such as the Vaccine Adverse Event Reporting System (VAERS) and the World Health Organization’s (WHO) Early AI-supported Response with Social listening (EARS) platform ( for Disease Control and CDC ; Purnat et al. 2021 ) . Unfortunately, many such platforms (including EARS) have been shut down or are no longer maintained, partly due to increased restrictions and pricing of social media data access ( Stokel-Walker 2023 ) and shifting prioritization of government policies and funding ( Stein and Stone 2025 ; Valentino 2025 ) . Motivated by the need to monitor concerns towards vaccines, we propose ConcernScope, a platform that can quickly and effectively identify vaccine concerns in a corpus. This application is intended to quickly and accurately classify large amounts of text data, summarize the results, and return vetted information relevant to the concerns. We further demonstrate additional applications, including identifying trends in vaccine concerns through time series analysis and examining shifts in concerns surrounding significant events. We build directly off of prior work on classifying the landscape of public health concerns surrounding vaccination, utilizing the VaxConcerns taxonomy ( Stureborg et al. 2024b ) , a disease-agnostic taxonomy of concerns that may drive people towards hesitancy. This taxonomy organizes concerns into two levels, one of broad granularity with concern categories such as Health Risks and another of finer granularity with specific claim categories such as vaccines having Harmful Ingredients or Specific Side-Effects . It is composed of 5 parent categories and 19 child classes each associated with only one parent category. To efficiently classify text using VaxConcerns, we fine-tune a BERT model by distilling labeled data from GPT-4, leveraging GPT-4’s high-quality annotations as a foundation for the task ( Xu et al. 2024 ) . We focus primarily on GPT-4 as prior work found it achieved strong performance among several closed and open-source LLMs ( Zhu et al. 2024 ) . Classification on the hierarchical taxonomy’s label-set can include multiple concerns in a single passage of text, and we therefore build on previous work on hierarchical multilabel classification task ( Liu et al. 2023 ) . Our models produce independent binary prediction (“present” or “not present” in the text) for every node in the VaxConcerns taxonomy. These predictions must be made for both the parent and child categories, since a text can invoke the broad category (e.g. Lack of Benefits ) with and without a specific child rationale (e.g. Existing Alternatives ). For example, consider the YouTube comment: “ I don’t need the vaccine! No reason to get it ”, which clearly invokes the parent label Lack of Benefits without invoking any of the child labels. In the remainder of the paper, we first discuss, in Section 2 , our method for building such a hierarchical multilabel classifier for ConcernScope. Then, Section 3 describes our web-based ConcernScope platform to be demonstrated. Finally, Section 4 presents a case study of 186,000 samples from 9 blogs that regularly attack vaccination as a practice, demonstrating how ConcernScope can capture trends in concerns surrounding vaccination over time. 2. Classifying Vaccine Concerns Classifying text to identify vaccine concerns is a difficult task even for human annotators ( Stureborg et al. 2023 ) , but LLMs have been shown to perform well under zero-shot settings when labels are well defined ( Stureborg 2024 ) . However, using LLMs directly for this task remains prohibitive due to computational costs and reasonable latency requirements ( Samsi et al. 2023 ) . Therefore, we explore an alternative approach that first uses LLMs to label a training dataset; then, using this training dataset, we finetune a less expensive model ( Hinton et al. 2015 ) (such as BERT) to be used by ConcernScope at runtime. Ideally, this knowledge transfer process with synthetic data generation retains the LLMs’ high performance while requiring much less computation. Although ConcernScope needs only one hierarchical multilabel classifier in the end, during the process of training this classifier, we employ a second vaccine relevance classifier. The purpose of this classifier is to quickly filter out text that is not vaccine-related from more expensive labeling of vaccine concerns. Both classifiers are trained using a similar approach, leveraging knowledge transfer from LLMs. The approach of using LLMs to create a training dataset for finetuning less expensive models requires highly accurate LLMs. In their technical report, OpenAI et al. 2024 show that GPT-4 demonstrates strong reading comprehension, as it exhibits human-level performance on many academic exams and advances the SOTA on several NLP benchmarks. Given our tasks’ complexity and the need for high accuracy, we therefore use GPT-4 for generating and labeling the training dataset in a zero-shot setting. 2.1. Data Preparation Our primary dataset is composed of 186{,}000 passages from various websites (Table 1 ). These websites have both articles attacking vaccination as well as articles not related to vaccines. From this dataset, we sample 500 passages and label each of them for vaccine

Read the complete text