Beyond Static Benchmarks: Synthesizing Harmful Content via Persona-based Simulation for Robust Evaluation

Huije Lee Jisu Shin Hoyun Song Changgeon Ko Jong C. Park · 2026

Read Beyond Static Benchmarks: Synthesizing Harmful Content via Persona-based Simulation for Robust Evaluation by Huije Lee Jisu Shin Hoyun Song Changgeon Ko Jong C. Park (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-17020.txt.

Introduction

Lee and four colleagues build a tool for making harmful content to stress-test detection systems, since fixed benchmarks scale poorly and may be contaminated by web-scale pretraining. Dual personas, blending backgrounds and interests with harmful strategies, drive diverse simulated interactions. Human and model judges confirm high harmful-generation success, and the synthetic cases prove harder to detect than existing benchmark items, with diversity matching human-curated sets.

Catalog record

CREATORHuije Lee Jisu Shin Hoyun Song Changgeon Ko Jong C. Park
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE62531
LICENSEhttp://creativecommons.org/licenses/by-sa/4.0/
RIGHTSCC BY SA
SOURCEarXiv

Source: https://arxiv.org/abs/2604.17020

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Beyond Static Benchmarks: Synthesizing Harmful Content via Persona-based Simulation for Robust Evaluation Huije Lee Jisu Shin Hoyun Song Changgeon Ko Jong C. Park Abstract Static benchmarks for harmful content detection face limitations in scalability and diversity, and may also be affected by contamination from web-scale pre-training corpora. To address these issues, we propose a framework for synthesizing harmful content, leveraging persona-guided large language model (LLM) agents. Our approach constructs two-dimensional user personas by integrating demographic identities and topical interests with situational harmful strategies, enabling the simulation of diverse and contextually grounded harmful interactions. We evaluate the framework along three dimensions: harmfulness, challenge level, and diversity. Both human and LLM-based evaluations confirm that our framework achieves a high harmful generation success rate. Experiments across multiple detection systems reveal that our synthetic scenarios are more challenging to detect than those in existing benchmarks. Furthermore, a multi-faceted analysis confirms that our approach achieves linguistic and topical diversity comparable to human-curated datasets, establishing our framework as an effective tool for robust stress-testing of harmful content detection systems 1 1 1 Our dataset is publicly available at https://github.com/huijelee/synthesizing_harmful_content . . 1 Introduction Online harmful content, such as toxicity and hate speech, fosters hostility and hinders healthy and safe communication. To mitigate these risks, various detection systems, including large language models (LLMs), have been developed 7 ; 9 . Typically, these systems are evaluated on public benchmarks composed of test instances curated by experts or harvested from online platforms 54 ; 43 ; 49 . However, reliance on such static benchmarks presents several limitations. Manual curation of test instances limits scalability, creating a bottleneck in scaling or updating benchmarks to keep pace with rapid model development 36 . Static benchmarks also fail to capture the complexity of real-world harm due to limited diversity. They are prone to lack topical coverage of newly emerging social issues 23 ; 37 and expressive diversity across tones, linguistic styles, and sophisticated malicious behaviors 1 . Consequently, models evaluated on such benchmarks may perform well on frequent patterns while missing rare but critical harmful behaviors. Publicly available benchmarks may also be affected by contamination from web-scale pre-training corpora 19 ; 11 , further motivating evaluation with fresh and dynamically generated scenarios. To address these limitations, we propose a framework for synthesizing harmful content, enabling robust evaluation of detection models. Instead of relying on a fixed corpus, our framework employs LLM agents that participate in real-world discussion threads to flexibly generate harmful content tailored to specific topics and styles, enabling evaluation against unseen and evolving threats. While synthetic data offers a path to scalability 22 , existing prompting approaches are prone to stereotypical, simplistic, and repetitive content that fails to reflect the diversity of human behavior 46 ; 30 . To improve diversity, we introduce a two-dimensional persona-based LLM simulation. Drawing on the observation that real users maintain consistent identities while adapting their behaviors across contexts 6 ; 2 , we construct personas along two distinct dimensions: intrinsic aspects (demographic identities and topical interests) and extrinsic aspects (situational interaction strategies). By simulating diverse persona-guided agents, our framework generates harmful content that is both contextually grounded and strategically varied. We validate the effectiveness of our framework through a multi-faceted analysis focusing on harmfulness, challenge level, and diversity. First, evaluations conducted by both LLMs and humans confirm a high harmful generation success rate. Second, we evaluate the detection performance of existing safety classifiers, revealing that our framework exposes hard-to-detect cases that are overlooked by conventional benchmarks. Finally, we conduct a comprehensive diversity analysis, showing that our framework achieves diversity levels comparable to human-curated datasets. A further detailed analysis confirms that the integration of two-dimensional personas is instrumental in enhancing both linguistic and topical diversity. Together, these results demonstrate that our framework provides an effective tool for robust evaluation of harmful content detection systems. Our contributions are threefold: • We propose a framework for synthesizing harmful content, leveraging two-dimensional user personas to facilitate robust evaluation of harmful content detection systems. • We demonstrate the effectiveness of our approach, achieving a high harmful generation success rate with diversity comparable to that of human-curated datasets. • We show that our framework serves as a robust evaluation tool by exposing hard-to-detect cases overlooked by conventional benchmarks. 2 Related Work Static Benchmarks for Harmful Content Early research on harmful content relied on static benchmarks 54 ; 18 ; 49 . These benchmarks were created by either collecting data from online platforms 43 ; 33 or through expert-led curation 8 ; 13 . Such datasets were instrumental in training models and later served as benchmarks for evaluating the zero-shot capabilities of large language models (LLMs) 35 ; 51 ; 15 ; 21 ; 9 ; 16 . However, relying on fixed benchmarks presents limitations. With the rise of web-scale pre-training, data contamination poses a significant validity threat, as models are likely encounter test instances during training 19 . Furthermore, manual curation lacks scalability and efficiency, struggling to keep pace with the rapid evolution of LLMs 41 ; 10 . Static datasets also fail to capture the evolving nature of toxicity, lacking the diversity to represent emerging social issues and nuanced harmful behaviors. Synthetic Data Generation To address data scarcity and evaluation rigidity, researchers have increasingly turned to synthetic data generation 55 ; 5 ; 47 ; 31 ; 50 ; 48 . 52 introduced a dynamic adversarial generation framework involving human-in-the-loop annotation to expose model vulnerabilities. Building on this concept, recent work has shifted toward fully automated generation using LLMs. ToxiGen 22 utilized demonstration-based prompting, while 46 combined jailbreaking prompts with few-shot demonstrations to elicit harmful content. More recent work, such as Toxicraft 26 , sought to enhance diversity by systematically refining topics and context from seed examples. Toxilab 25 fine-tuned open-source LLMs to narrow the quality gap with closed-source counterparts. Despite these advances, generating diverse harmful content remains challenging. Safety-aligned LLMs frequently refuse to generate harmful content, and successful outputs typically exhibit stereotypical or simplistic patterns, lacking linguistic and topical variance. Persona-based Simulation Recent studies in agent-based simulation have demonstrated the potential of LLMs to model complex social dynamics and realistic user behaviors through persona instantiation 40 ; 39 ; 17 ; 45 . While these studies focus on general social interactions, we adapt this paradigm to safety evaluation. Beyond utilizing personas defined solely by demographic attributes, we incorporate elements representing behavioral strategies to construct contextually rich and varied harmful scenarios. This approach draws on the observation that real users maintain consistent identities while adapting their behaviors to different situations 6 ; 2 . Consequently, we conceptualize persona design as comprising intrinsic aspects (inherent identity and interests) and extrinsic aspects (interaction strategies), enabling the generation of diverse and sophisticated harmful content. 3 Method In this section, we present our framework for persona-based harmful content generation. Our framework consists of two components. First, we synthesize user personas for simulation. Then, persona-guided agents interact within online community threads to generate harmful content. 3.1 Persona Design In our framework, a persona is a structured characterization of a potential online participant. This profile details a consistent set of user characteristics, including identity features and behavioral tendencies, that shape how the participant may engage

Read the complete text